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OVERSIGHT HEARING ON THE NATIONAL AS- 
SESSMENT OF EDUCATIONAL PROGRESS 
(NAEP) 



WEDNESDAY. MARCH 13, 1991 

House of Representativss 
Subcommittee on Elementary, Sm)NDARY, 

AND Vocational Education, 
Committee on Education and Laror, 

Washington, DC 

The subcommittee met, pursuant to call, at 9 a.m.. Room 2257, 
Raybum House Office Building, Hon. Dale E. Kildee [Chairman] 
presiding. 

Members preront: Representative Kildee, Martinez, Sawyer, 
Goodling, and Gunderson. 

Chairman Kildee. The Subcommittee on Elementary, Secondary, 
and Vocational Education will come to order. 

The subcommittee meets today for the first of several hearings 
on educational testing. Today's hearing focuses on the National As- 
sessment of Educational Pn^ress, 

As the only federally mandated assessment of student achieve- 
ment, NAEP has become the focus of much attention in the nation- 
al testing debate. Today's witnesses bring a high level of expertise 
and will advise us regarding NAEP's importance as a national as- 
sessment tool and what changes the committee might consider in 
the future. 

Today's testimony will also provide a foundation for Lomorrow's 
hearing when the subcommittee will examine the pros iJid cons of 
national toting generally. 

Certainly the question of what, if any, NAEP's role should or 
will be is an integral part of this discussion. I had hopt^d that the 
administration would provide a witness this morning to explain 
how NAEP is administered, and while the Department of Educa- 
tion declined at this time the subcommittee's invitation to t^tify, 
it is my understanding that Secretary Designate Lamar Alexander 
has a greftt deal of inter^t— we know that— in educational testing. 

I am informed that he would like to testify on NAEP himself at 
some later point, and I look forward to that testimony. 

,My opening statement will be brief this morning. I turn to my 
friend and the ranking Republican member of the committee and 
also the ranking Republican member of the full committee. Bill 
Goodling of Pennsylvania. 
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Mr. GkK)DUNG. Thank you, Mr. Chairman. I would like to com- 
mend you for calling the hearings today and tomorrow on the im- 
portant topic of educational testing and assessment. 

With national goals, national tests, and national curriculum, and 
last year the big kick wap national certification of teachers— all 
these great things we are going to do to improve education— being 
discussed seriously acro^ the country, it is important that this 
committee become actively involved in this area. 

I would like to put into the record, and read it into the record, a 
statement of my secretary of Education back in the State of Penn- 
sylvania in a letter that he wrote, and I won't say io whom he 
wrote the letter. 

"If the speed of the invitations to give testimony on national test- 
ing indicates interest, as you suggest, it also indicates a pell mell 
rush to jump on a band wagon. I do not have time to develop my 
response t^yond the following bullets but wish to have my con- 
cerns on record. 

The principles of r^tructured education— flexibility, changed 
student^teacher roles, cla^room based management, et cetera— 
cannot be accomplished for the long term cast against a national 
toting program. I don't care how sensitively it is constructed and 
administered; it will ultimately drive our education train, not just 
guide and inform it. 

There is in my mind little or no relationship between the im- 

?rovement of student learning and a national testing program, 
here is a relationship between student performance on national 
standards and a national testing program, but they are not the 
same thing. National testing is the wrong way to go if we believe 
that schools are places for learning, not sorting kids into successes 
or failures. No matter what is intended by such a program, I be- 
lieve one inevitable result will be to place academic value signs on 
individual children. The attached article states many of my 
thoughts on the issue/' 
I just want to read one or two of th<^. 

**Ve8, everyone is concerned about the quality of our schools, and 
y^, everyone wants more accountability from our schools, but 
there is already a multitude of national standardized tests. How 
will yet another national exam help improve elementary and sec- 
ondary education? The Ford Foundation reported recently that 
mandatory testing consumes some 20 million school days and the 
equivalent of $700 million to $900 million in direct and indirect ex- 
penditures annually, an enormous cost and use of classroom time 
that could be spent on skill development. 

Here are seven reasons why national tests won't improve our 
schools. One, they do nothing to help students learn. Two, they are 
poor indicators of student performance. Three, they provide no in- 
depth assessment of the curriculum in a particular school district. 
Fr>ar, they encourage teaching to the test. Five, thev categoiize and 
label students. Six, they are racially, culturally, and socially biased. 
Seven, they measure trivial information easiJy tested by multiple 
choice." 

I would tell you. Mr. Chairman, this Secretary has a pretty good 
reason for being rather concerned. His predecessor in my State had 
the grand idea that we would test all the public school children. 
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and then we would rate the schools according to the test results 
and come up with this magnificent idea that a w^hool district in 
western Pennsylvania is number one in the State of Pennsylvania. 

Of course, I supervised student teachers in that school district 
many years ago. If it isn't number one, they sure aren't getting 
their money 8 worth because, number one. they have probably 
double per pupil expenditure than many other school districts in 
the State of Pennsylvania. 

This isn't the fault of a State formula. It is just that they add to 
what they get from the State and are willing to pay, and can easily 
pay much, much more. 

Then, secondly, every parent is a college graduate, many of 
whom have Ph.D. s, and thirdly, those children by the time they 
open their eyra or even before, the demand that the parents place 
on excellence from their children as far as education is concerned 
and from the schools—I mean, it is just automatic sp that is why 
you would get his real concern about whatever it is we may be 
doing as far as any kind of national test is concerned. 

Sorry I took that much time, but I thought it was important, I 
may not be here tomorrow to get all those comments in. 

Chairman Kilj>E£. No, I appreciate very much your opening 
statement, Mr. Goodling. It is very helpful to the committee and 
our own personal conversations have always been very helpful to 
me. 

Our panel this morning will consist of Dr. Herbert J. Walberg, 
board member. National Araeeement Governing Board, Chicago, Il- 
linois; Dr. Daniel M. Koretz, Senior Social Scientist, RAND Corpo- 
ration, Washington, DC; Dr. Robert L. Linn, Professor, University 
of Colorado, Boulder, Colorado; and from my own State of Michigan 
Dr. Edward D. Roeber, Supervisor, Assessment and Accreditation, 
Michigan Department of Education, Lansing, Michigan. 

If they would step forward, and if they would testify in the order 
in which I called their names or any order they may have devised 
among themselvra. 

Dr. Walberg, though, will lead off. 

STATEMENTS OF HERISERT J. WALBERG, MEMBER, NATIONAL AS- 
SESSMENT GOVERNING BOARD* CHICAGO, ILLINOIS; DR. 
DANIEL M. K ;RETZ, SENIOR SOCIAL SCIENTIST, RAND CORPO- 
RATION. WASHINGTON* DC; DR. ROBERT L- LINN, PROFESSOR. 
UNIVERSITY OF COLORADO, BOULDER, COLORADO; AND DR. 
EDWARD D. ROEBER, SUPERVISOR. ASSESSMENT AND ACCRED- 
ITATION, MICHIGAN DEPARTMENT OF EDUCATION. LANSING, 
MICHIGAN 

Dr. Walberg. Mr. Chairman, let me express my appreciation for 
the opportunity to testify before this committ^. l would like to re- 
quest that my full written testimony be made part of the record. 

Oiairman Knj>BS. Yes» without objection you may summarize in 
any fittshion you wish. 

Dr. Walbbrg. Thank you. I will try to be brief, but I have been 
asked by staff to give an overview of the National A^esament^ and 
I will tiy to do that as quickly as I can. 
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As you indicated, my name is Herb Walberg. I am a professor of 
education at the University of Illinois at Chicago, and one of my 
areas of specialty is t^ing and measurement. 

I tiiink, as you know, tne National Assessment is our only na- 
tionally representative assMsment of what American students 
know and can do. It has a 20-year remrd of providing what we 
hope is useful information to educators, the public, parents, and 
others that are concerned about how our nation is doing in terms 
of the standard school subjects such as reading, mathematics, sci- 
ence, and even adult literacy as well. 

My remarks this morning will give something of an overview* 
but 1 would like to stick pretty much to lK>ard policy. If any ques- 
tions come up, 1 would be glad to give my own personal view, but I 
would like to reprint what our board has been doing and what 
we have been considering. 

The main purproe of the National Assessment is to improve our 
nation s schools by making objective information available alx)ut 
student performance in elected learning areas, available to policy- 
makers at the national, regional, State> and local levels. 

We report on students at roughly ages 9, 13, and 17 which corre- 
spond to grades 4, 8, and 12. I think it is important to recognize 
about the National Assessment that not onlv has it been providing 
this information, I think, with considerable utility to educators, 
policymakers, legislators, and others, but it does this in a way that 
minimizes testing time. 

Although people think sometimes that the National Assessment 
involves mass testing, there are design features in the National As- 
sessment especially having to do with sampling that enables us to 
restrict this to very small numtei^ of students, and we have never 
proposed to do mass tests of a whole State or the whole Nation. 
Other groups are now proposing that, but we have never done that. 

Lik? national public opinion surveys such as the Gallop Poll in 
which, SLy, about 1500 people can provide a reasonably good esti- 
mate of what Americans are thinkmg, we by taking small, repre- 
sentative samples can make a very good estimate of what our stu- 
dents are doing from a national standpoint. 

We also make considerable effort to minimize the student testing 
time for an individual student. We do not take more than an hour 
of any individuals test time so we have a considerable amount of 
effort to do that. 

One of our main roles, unlike almost any other testing program 
in the United States, is implied in one of the terms that we use, 
"progress/' We a^e very interested in measuring national trends 
over time. That enables us to find out if students are improving or 
not improving, what kinds of students, and in Uie past we had re- 
gional assessments of different parts of the country. 

As you know now, by Congressional legislation we are testing 
specific States that want to participate in the program on a strictly 
voluntary basis. Roughly 40 States participate on that voluntary 
basis on the first round, and in the second proposed round another 
40 have also voluntarily chosen to participate in the new assess- 
ment. 

The role of the National Assessment Governing Board is to be 
the policymaking board. The board sets policy with respect to such 
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things as the frequency of the as^ment, the subjects that will be 
test^, and certain design feature. 

There is a three, or tripartite— it is a governing structure m 
which the board sets policy. The National Center for Educational 
Statistics as a part of the United States Department of Education 
gives oversight and monitoring of that policy, and then there are 
contract grants and other contractual arrangements. The vendors 
carry out the actual work. Under the 1988 reauthorization Con- 
gress created this tripartite structure. 

One of the things that I think is particularly important and may 
be of interest to you as you consider national testing, particularly 
with respect to the National Assessment, is the independence of 
the bc^rd. That has been a matter of some concern to us. 

The board, in its Congressional foundation, emphasized that it 
was an independent board. It (insists of two governors, two State 
l^islators, tnree teachers, a Mcondary school principle, an elemen- 
tary school principle, a local and State superintendent, two curricu- 
lum specialists, two toting specialists, one non-public school ad- 
ministrator, a representative of business and industry, and three 
repi^entatives ot the general public so it is very mi!ch of a general 
board, and the idea of this board, we think, in the Congressional 
legislation was to keep it separate from the Department of Educa- 
tion and various kinds of special inter^t groups that might have a 
special agenda with respect to national curriculum. 

Two of the chief things that we have been working on since the 
inception were mandat^ by the Congressional legislation- One of 
the most important is what we call a consensus process. 

In order to come up with an examination that is fair to students 
in different parts of the country, we try to rather tc^ther groups 
that would nave a special expertise: teachers, other educators, 
people in business and other fields, that formulate a prescription or 
goals about what should be taught, and this is a very painstaking 
process. It takes a lot of information and analysis to look at all the 
new curricula that are being tried out in American schools and 
come up with a consensus on what ouirht to be in that test. So it is 
a very painstaking process, and we have been working verv careful- 
ly to make a faithful representation of what is actually being 
taught in American schools. 

"nxe other big effort that we have been making since our incep- 
tion is to set achievement levels, and a^ain this was by— we were 
asked to do this in the Congressional l^islation. 

In the past 18 or 19 years, the National Assessment simply re- 
ported on how students were doing such as the averages of stu- 
dents in various parts of the country, but for the first time we have 
been asked to set stantisirds and so we have gotten groups similar 
to this consensus type groui», educators and people from other 
walks of life in the United States, to say not only how the students 
are doing but how well they are doing, so for the first time we will 
have national standards. 

We have tried this out for the first time so we will be able to say 
what percentage of our students are actually proficient— proficient 
meaning, in general, that they are able to go on to the subject 
matter of the next grade level— so we will be able to say how many 
of our students are able to do that, taking it somewhat beyond 
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that, and 1 could go into more detail later if that is of interest to 
you. 

We have conducted particularly this first round, and especially 
with respect to State comparisons very much on a trial basis. It is 
called trial in the l^islation, and we are doing this as an experi- 
ment. We are trying to get throiigh the voluntary 40 States that 
are participating an ^timate of how each State is doing, and then 
we will be comparing the States, not only with respect to the aver- 
ages of the different States but with respect to whether they meet 
these standards. 

We hope that this will be useful to people in local school dis- 
tricts, to States, and in the Nation, particularly not only to know 
how well they are doing but what characteristics of their programs 
may be accounting for the difference between the Stat^. 

To give an example, if we had 10 States that had, let's say, effec- 
tive homework policies, for example, or particular types of teacher 
training programs, we would like to know how well those States 
have done in comparison with other States. We would not be able 
to conclude with certainty that certain policies caused those things, 
but people would provide one source of information on whether 
some of their policies appear to be effective or not. 

And as you know, because of the extraordinaiy interest in educa- 
tional reform and since 1983 with the publication of A Nation At 
Risk, many States on a State basis are trying some extraordinary 
policies of one kind or another, and they would like to see how 
they would rank, so that is the interest in these types of compari- 
sons. 

We would like to take two further initiatives, and perhaps might 
be considered in further legislation. This is fully described in the 
documents that I have submitted to the committee, but we A^ould 
like to authorize further State assessments. 

We have done the first — we have plans, conducted the first as- 
sessment and the second has been authorized, but we would like 
authorization, smce there has been a keen interest in this, if— and 
I should mention, too, that our work is being carefully evaluated 
and assessed not only but by various other groups, the National 
Academy of Education, the >fational Center for Educational Statis- 
tics, and there has been a lot of interest in the scholarly communi- 
ty and the public policy community in how well these things are 
doing — we would like authorization if these steps that we are 
taking to make State comparisons are successful, we would like to 
continue them further and continue to improve them. 

A second point that we would like to bring before you is the pos- 
sibility of conducting the National Assessment on an annual basis 
rather than a biennial basis. We think because of the extraordi- 
nary amount of interest, certainly in Congress but also by the edu- 
cational summit of President Bush and the governors that have 
stated national goals, that we need to have this testing more fre- 
quent so we can trace Sf)ecific progress from year to year. 

Also we think, on the board, that we need to clarify the toard s 
independence. There is explicit language in the authorizing legisla- 
tion that the board is to, and I quote, ''formulate the policy guide- 
lines for the national ass^ment'* and to, quote, **exercise its func- 
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tions, powers, and duties independent of the Secretary and other 
offvxs and ofHcers of the Dei»rtment of Education." 

Yet the legislation also says that under the name legislation the 
Commissioner of Educational Statistics carries out with the ad>ace 
of the National Assessment Governing Board, and we think that 
there is a discrepancy there as to whether we are truly a governing 
board as our name implies, or whether we are merely advisory. 

So we would like for the committee to consider those factors. 

Well, I have perhaps taken a bit more time than I should. I think 
some of these issues are quite fundamental. I would be pleased to 
answer any questions that you have, either now or in writing later 
or whenever the chairman is pleased. 

[The prepared statement of Herbert J. Walberg follows:) 
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Hr Chairman, lot mo rxpross »y appreciaticn for the opportunity 
to' testify bofor« tho Subcoa»ttt©o on Bl««ent*ry, Secondary »nd 
Vocational Education about tha National A&sGSSBont of Educational 
Progress (HAEP) * 

My name is Herbert J. Walbor^. I havo boon a TOabor of th* 
National AsscBsacnt Governing Board (tho Board) ainco ita incepti<m 
In < ctober of 1988 and a« chairsan of its Design and Analysis 
Cowittea. I am alao professor of education at the University of 
niinoisi one of »y areas of specialty is teatin^ and Keasurenent. 

NAEP, in its twenty-two year history has developed a well deserved 
reputation for integrity and innovation. Nov known as "The 
Nation's Report Card," NAEP is our only nationally rf?presontativa 
asfiessBsent of what A»erica*8 students know and can do* Also« as 
part of a pilot project authorized by the Congress in 1988, NAEP 
will report in June 1991 for the first t ioe State-representative 
results froa the 1990 »atheaMitic» assessment* 

I AB honored to provide testimony on this osteeaed federal proflraa. 
However X would want to emphasise that »y roMrks absmt possible 
changes to the HktV legislation will be linitod to policies adopted 
by the Board. As you know, the NAEP legislation specifies the 
responsibilities and functions of th«i Board* Tha law also 
stipulates that "...tha Board, . .shall be Independent of tha 
Secretary (of Education] and the other of f lce» and officers of the 
Depart Bont of Education." Thus, wy renwirka will not necessarily 
represent s>osltlons of the Dopart»ent of Education. 

This Roming l vill provide an overview of NAEP and the role of the 
Boards discuss the utility of S*ate-representatlva data, and 
describe Board policies that require leqiolative action. 



OVSfi7ZSW RASP 

The purpose of NAEP Is "...to i^rovo our Nation'a schools hy 
Baking objective inforaatlon about student perfomanc© in selected 
learning arsas available to pollcyaakers at the national, regional, 
state and local levels** 

Omisr current lav, the National Assessment is conducted every two 
years. The subjects covered and their prescribed cycles are« 
reading and aathoaatics, at least once evory two yearef trriting and 
science, at least once every four years i and hiutory/geograpby at 
least once every six years. Tha lav also pemits the Board to 
select other subject areae fcr assesraaent, including assesesents 
of adult literacy. 



1 



ERIC 




10 



The National A«i8ossi&cnt report ults on students at agos ninm, 
thirteen and seventeen and i.. ^.adaa tour, aight and tvaW«. 
However, it is Important to point out that sinliiiaing test burden 
is an overriding HAEP policy. HAEP does not test all students at 
tho^o ages and grades^ it tettts only a small »aBpl«« F0r axaiq;>I«, 
of th<* iwsro than nine million students in the three grades covered 
by NAFP, approxlniatoly ^1,000 participated In the nevly developed 
1990 reading as^sessmcnt and approximately 2.000 students per Stata 
participated in tho 1990 eighth grado matheoiatics Stato trial 
assesssent . 

Another way N^EP lislta tee^t burden is by minlaizlng student 
testing tiBH>, Testing time under HASP is ak>out one hour per teoted 
*%t\Klent. Thi'A is .ifcnnplished by a procedure knovn as satrix 
-..iiapl ing . ^ .t rtvttrlx Sf^wpHng, tHjuivalent subeawples of students 
r*>oi-?v<» 'iiffermt poitions of the total pool of test questions* 
The total 8tud*-nt KAsple is lajg© enough to estisate precise 
results for each question. sinca NAEP is intended to report 
aggregate student reKults at the national and regional levels and, 
umier the trial State aBs;essmcnt, at the Stato level, this is an 
ofticlont way to assrfss student performance. 

One final aspect of NAEP should bo highlighted — its capacity both 
to report trends over time and to incorporate nev developnents In 
curriculum and in assessmert technology. To do this, HAEP conducts 
parallel assessnonts, one lor trend data and one that is "cross* 
soctioral," to reflect current thinking and practices. For trends, 
KAEP \ses proccsiures and itess from past assessTOntsf this 
maintains what is now a twenty -year trend lino in subjects such as 
readi;ig and matheiaatics. Tho cross*sectional assessment reflects 
cur«.ent research and best practice in each subject area Idontifiad 
through a national consensus process that is aosinistered by tha 
Board. X will be talking sore about this consensus process later 
iii my testiaony. 



ROLS Oy TBB NATIOyAL ASBBaSKEKT GOVBRBXIfO BOASU> 

In 198B the Congress reauthorieed NAEP, staking iBtproveaents* 
providing for the stato trial assessoents, requirif>g evaluations, 
and redesigning the governance structurs to include an independent 
Board to forsalattt policy for KABP. 

Prior to 196S^ NAEP vas conducted under a gra- froa the E>epartMnt 
of Education. The pr9»l966 liAEP authority * ^luired th« grant** t9 
establish As&esswont Policy Conittec <} to advis* on policy 
aatterv. Thin arrangooent vas intm^ed to insulat* 1CA£P policy 
developBont and the conduct of th« ftasos»ont fro* th« D^Ntrtsent 
of Education* But it also had the effect of ispoding tlis 
Departaent^s ability to assure accountability In the admin ist rat ion 
of the MA£P program* 



n 



Under tho 1988 NAEP r©a«thor l««tlon, th« Con^rcM created a 
tripartite structure for HAEPi an independent governing board to 
formulate policy for the »»»efts»entf administration of the RABP 
proorae within tho National Center for Educatiof> StetieticsF and 
conduct ot HASP through contracti, grants or coojwretive 
agreements. The result Je a syetea of chocks and balances that 
permits the Depart»ent of Education to rosain at ••arwe length* fro» 
testing policy but allows appropriate accountability for prograe 
adainiftlration- 

The legislative history of the current BAEP legialation indicate* 
that tho independence of the ©card waa an ioportant consideration? 
both the Senate and Conference reports discuss provisions that have 
this parpose. At least three provisi<Mis of the law appear to be 
d i rect chI at the i ndf»pend«7nco of the Board f speci f ically , tho 
provision «lrpady mentioned regarding tho Board's independence of 
the SiHrretary; the rccjuirement that the Board •'exercise its 
independent judgment free from inappropriate influences and special 
inter sts»; and the roquiretBOnt that appointments to the Board are 
made t ' the secretary of Education only froa candidates nominated 
by the Board, 

In addition, the composition of the Board as prescribed in the law 
seems to be founded on the idea of an independent Board. The 
Board has tventy-three voting members: two governors and two state 
legislators {both on a bipartisan basis); three teachers; «n 
filosentary and a secondary tKrhool principal; a local and a Stats 
school board member? two chisf state school officorsi one school 
superintendent! two curriculum special lets i two testing e>fperts; 
one nonpublic school administrator; a representative of business 
or industry; and three representatives of the general public. The 
only federal member, the Assistant Secretary for Educational 
Research and Improvement, corvos in a non*votlng capacity. 

It is difficult tr imagine a group of individuals who, by the 
nature of their j? ivate roles, would be worm grounded in the 
prlnclplo of State and local primacy in education matters and less 
inclined toward federal intorventlonlom. 

He believe that a 1987 report about NAEP was considered by the 
congress whsn it toolt up the 1980 NABP rssuthorisstion. The report 
of tbs Study Group headed by then Governor liSBsr MsKsnder snd B. 
ThoMS Jaaes — mntltlsd •Thm nation's Rsport Csrd«--r«c^Mdsd a 
tripartite govemancs structure •lailar to that in currwit law* 
With rsspsct to the body that would bs rsspofMibl« for HASP policy, 
the Study Group recoasended that; 

Ths govsrwancsr and policy dirsction of the national assessment 
should b* furnished by m broadly reprmsmntativs IBosrdl thst 
provides wisdos, stability snd continuity^ that is c^ftsrssd 
with meshing ths sssesnsnt nssds of states end locslitiss 
with that of ths nationi that is socountsbls to the piAlie-- 
and to ths fsderal govemsstt— for stewardship of this 
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iaportant activity; but that is ltsf>lf bufti^rc<3 from 
iDanipulat ^ '^n by Any individual « level of government , or 
special interest within the field of education* 

With this as background^ t would llHo to discusB some of the 
specific responsibilities of the Board as &et out in law. 



rhe general responsibility of the fk>ard is to forjwjlate the policy 
guidelines for the National Assessisent. This includes fonoulating 
policy for; the methodology of the assessment; analyzing data and 
reporting results; selecting non-mandated subjects for assessment: 
and improving the fom and use of the National Assessisent. 

!ri addition^ the Board has responsibility for two activiticr* that 
are relatively large in scope and important to the conduct of NAEP. 
The first of these, forming the foundation for each assessment, is 
t^ie work of developing consensus on the content for each 
.isstfHSBient . The second, identifying achievement levels for each 
grade for each assessment, in fact establishes benchmarks for 
interpreting the results. 

CQns9.n^.u§^Tr5c?§i*-,9n Test Cgnteni 

twel oping consensus on the content of each assessment involves 
much work. With a decentralirod public education system of 16,000 
r.chool districts and fifty-odd state education agencies (including 
the District of Columbia and the outlying areas), the need tn 
include non-public schools as well, and the importance of 
incorporating the perspectives of researchers, practitioners, and 
<;ub^ect area special ists, arriving at a consensus in each subject 
area on what should be included in nAtP is an immense and 
complicated undertaking. It involves literally hundreds of people, 
}s i'ompleted over a period of twelve to eighteen months, and costs 
approjciii^ately $430,000 per subject area. 

The result of this consensus activity is a document chat, for each 
grade, describes in very specific detail what the test will 
include. This document is used by the KAFP contractor to develop 
the tost questions that will be used for the assessment m that 
subject area. 

I.ike-wise. setting achievement levels for each assessment is a large 
and ccBplicated undertaking, and at the present time, the subject 
of intense scrutiny. Setting achievement levels, or defining what 
students gh£>ill5l know and be able to do in each grade and subject 
tested under NAEP, was required for the first time under the 1983 
»ALP reauthorization. Unlike consensus on test content, setting 
Achievement levels has never previously been done. 
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Realising that it in unchartod tarritory, tha Board has 

attapptad to procaed ^MrtKlantly. Hhm Board dacidad that ita firat 
effort vould ba a pilot linltad to ena aubject^^in this cati^ 
nathematica as part of tha 1990 assaasaont. Tha Board has obtstnod 
ths advica of tha forasost axparts in tha fisld of aducation 
BtandArd<»sottin9 to design tho aathodology for Getting achicrvesont 
levels* Tha actual «fork of setting tha ftchievesent levala involves 
the participation of both educators and non^educators . In 
addition, thm Board has conducted three public hearings to receive 
comant at various points in tho procooa, 

currently, tho Board is conducting a validation/replication study 
of tha initial vork on achievement levels for tha 1990 sathesatics 

assesQiBont. And concurrc^nt vlth these activities is an external 
€»vAluation of the process. Overall » setting achievement leveJs 
will involve hundreds of individuals ovor a ^riod of about twelve 
jnonths at a cost of as^rcxloately $350,000. 

The result of this first dchiovemont level -setting activity will 
be a document that describes vhat students should know and be able 
to do in sathesatics at the fourth, eighth and twelfth grades that 
will be used, on a pilot basis, for interpreting results on the 
1990 sathenatica a&sess&ent. Called for in the congressional 
legislation, ve intend to build upon this ei^riemre so that this 
atsbitious experimental effort can be iaprovod in subsequent 
assessments* As a conseguencs of our experience thus far, ve are 
already developing ways to isprove the process. 



tmSP RBAOTBOBIIATXOII ISSUBB 

Over the last fifteen months, the Board has adopted a number of 
policies that vould require legislative action. These policies and 
the rationale for each are described in detail at attachment A. 
t would like to highlight four -kf those policies. 

The first two issues address a conmon these ; use of naep for 
reporting results at the State and below the State (i,e, district 
and school level). The Board views these two issues differently, 
and X will discuss each in so»e detail. 

The Board iwlisves that $tata*represantatlvs and school and 
district -representative results, properly used, can bo vary helpful 
in informing policy and in iiiproving inatruction. 

For axaapla, in addition to average overall acores for tho 1990 
eighth grade satheaatics State trial assessBsnt, there vill ba 
aubscores by content araa (a*?* nu^»ers and operations^ 
jsoasurenent^ geoBotry* data analysis, and algebra), in addition, 
background guest ions in areas such aa tocher certification, 
insorvica training, instructional practlcas, and parental 
involvesent are also part of tho aasessiDsnt. 
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Thua< a state superintendent of instruction oight coiaparo Stat© 
results on tho subscales viUl thoca ©f a aiailar or naiqhborinq 
Stata, ana at tha sane timm look at teacher certification and 
inscrvice training data. This Kind of comparison can help identify 
strengths as vail as iroakn«BB©s, identify States that «ay hav« 
useful models, but moat l&portantly, it vould provide information 
on which sojse action can be taken that would be aimed at lBprovin9 
the education received by the students. 

The Board is not in favor of cocparison© for comparison's sake, and 
has adc^ted policies for reporting State results that would avoid 
the inclination to nisuae or sisinterprot resulto through simple 
ranking of State scores. 

1. Authpriilna St^ite"reprc??entatiY.«. !?^^i^<.'»^"»<'nt5. 

Congrrss authorized NAEP trial State assessiMnts in eighth grade 
mathejnaticB in 1990 and in fouith grade readlj>g and fourth and 
eighth grade i»athematics in 1992. No subsequent State testing is 
authorized. These trial State asfiCf;stBentfi are being closely 
p\on itorcxi by the CM^partn«>nt of Education, the NAEP contractor and 
the Board and will be the subject of analysis across tho country. 
A congressional ly required independent evaluation is being 
conducted by the National Academy of Education? reports are 
expected in March and September 1991 and in late 19»3. 

On Juno a, 1991, the National Center for Education Stat 1st lea will 
release data fros the 1990 State trial assesssont in eighth grade 
mathcnatlca, along with national results for grades four, eight and 
twelve. Al»ost forty States have already signed up for tha 1992 
State trial assessments and thc^e are likely to bo more* Unless 
tho evaluation proves that the Stats trials do not yield valid, 
fair and accurate data, and unless tha Stat« interest in 
participating evaporates, the groundswell of support for regular 
Statfi-repreoentative data as a part of NAEP la likely to c<mtinua. 

The Board believes that, given favorable results from tha 
congress ionally required evaluation, the NAKP legislation should 
be amended to provide a general authority to conduct State 
representative assessments, with ths frequency and subjects 
determined according to Board policy, with the full costs borne by 
the fedmrai govammeftt and vith Stata participation voluntary* 

2. Refltorino local optiona to ttg f KftEP 

Tha Boaixt bel lavas that tha NAEP lagisXation should be amandad to 
permit Statasr acTrool diatricta aiMl mm^public mcOioQlm to uaa NASP« 
at thair option and cM>mt, to oollmet data and r^ort on parforvaooa 
of schools and school dlatrict*. Thim would ramtora tho option 
that vam alivlnatad under tba i«sa raauthoriaation of RAFP. 

It ia not tba policy of tha Board that reporting bolow tha Stato 
level ahoifeld bo a fmftoral activity. Tho Board horn navar auggastod- 
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-nor dooa it now—that tho pr9ani£atlpn that conduct* NAEP should 
ostAbMsh saviplf^ In individual schools and schOQl districts 
part of tho KAEP contract and report the results. Tho Board view* 
this as a local control issuer-just as no stat« or district should 
b« rcH|uired to participato in HA£P^ noithor should a state or 
district be pr«>vented froiB usin^ HAEP to assess and rt'port &tud«>nt 
pcrforvmnco in its schools* 

Tho Board b6l loves it would bo better to give HASP on a rotating 
schedule of three subjects per year rather than ths current 
practice of testing three to five subjects every two years. Annual 
NAEP testing and reporting are noCrnsary to provide timely and 
sufficient data for po] i<-ymaKers and the public. This would allow 
NAEP to provide >cey peaRwres of acadosic ach i c>v(>ment for the annual 
progress repoita on Aret»rican education, called for last yoar at tho 
Education SuBUHit between President Bush and the nation's Governors. 

An annual NAEP would pioduce little net increase in test burden 
because the currrnt evety-other-yoar cycle includos up to five 
te&ts each testing year and extensive pre-testing in off-years. 
NAEP conducted annual testing fro^ 1969 until 29ao« so this 
proposal is not without precedent. 

Tho Board believes a permissivo authority allowing annual testing 
rather than a nandatod biennial cycle would provide the flexibility 
for NAEP to be responsive to inforsuition needs as they evolve. Of 
course, the number of subjects actually assessed would be subject 
to the availability of appropriations. 

L._Cl«iriljana,.tlit j;:Qlfi_ftl Jthe .Bear^,. 

The Board believes that, with respert to its rolo and 
responsibilities, the NAEP legislation contains asbiguities and 
conflicting provisions. For example, the NAEP legislation gives 
the Board the authority to ^foroulate the policy guidelines for the 
National Assessinent , * to ''exercise its functions, powers and duties 
independent of the Secretary and the other offices and officers of 
the OepartBent of Education,* to "exorcise its independent 
judgstont, free fro» inappropriate Influences and special 
interests,* and to *bire its own staff. " 

Taken together^ these provisions could not bo sore clear in their 
intent to establish an independent policy body for NAEP insulated 
fro« the jurisdiction of the Depart»ont of Education. However^ 
other provisions reduos this cl&rity. 

Under th» NAEP legislation^ th* CoBwissionsr of Educsti^ 
Statistics carries out NABP »(w)ith thm Q^tdSSi of ths ffationsl 
AsseasBont Governing Board {eta|>haBis added|** This hAs been 
interpreted by the Dspartsont of Education to ftoan that th« Board 
is an advisory conittee whose policy decisions oust only bd givsn 
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consideration by thm SocrotAry, Another p9vt of th^ lav provido* 
that *Tbm Sacratary Bay appoint , at tlMi direction of tha Board* up 
to m%% tadmical ai^>loyaos under sn axcaptad aanrieo aroointing 
aut?%ority. Tftia provision haa tha potential for aroding tha 
Board* a independanoa and contradicta tha ^rc^iaion that tha Board 
•hail hira ita Qvn staff* 

Anothar iasoa ralataa to tha Board'a hud9ot« vhich aa a set-aaida 
vithin tha NAEP lina itoa in tha Dapartaont'a Assoa9»ont« 
Statiatica, Research and taprovaMnt appropriatim, ia in fsct tha 
$acratary*a raaponaibil ity to adainistar. The c?ontrsdiction ia 
painfully cMbvioua— ^independanca froa tha Deparlaant ia incoi^tibla 
vith tha 8acratary*a budgot raaponsibility. 

While I hav« highlighted a few of the areas of concern with respect 
to dB^iquities In the rolo of the Board, attachment h contains a 
Boro coDplata anAlysis. However, I would point out that while we 
have not slvaya agreed, for the aost part, these issues have been 
handled openly and with a general spirit of cooperation with the 
Departaent . 

But these isBues are fundamental . Their existence erodea the 
principles in tha law on which tha Board vaa *atabliahodi 
Independence froa the Depart sent of Education, tripartita chec^ and 
balance HAEP governance atructure« and freadoa fn»i inai^ropriate 
influences and special interesta. Tha Board's only goal in thia 
analyais ia no aore than what it ^ievea tha Congress intended— 
a truly independent Board whose po^iciea direct tha conduct of 

Thank you vary suchr X wouid be pleased to answer any questions you 
Bay have. 



Attachsent Ar Policies Adopt ort by the Board ??<>q\iirJng I^glfilative 
Action 

Attachsent Bt Stateaent on the Future of HABP 
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RACKGROUNO 

During the past year, the National Xssc^sBont Governing Board 
adopted tvo cosprehon^ive policy statesents. In D«cfi»t>or 1989, th« 
Board adopted positions on th« Putur* of the national Asi«QdaMnt 
addressing nine specific policy issuM. In Hovesber 1990, th* 
Board adopt«d a statesiont on policy issue* for the 1994-96 KABf» 
contract avard addreseing twenty->fottr specific policy issues. 
Some, but not all, of those policies csnnot be iepleeented without 
amending the current NAEP authorisation. this report has bp^n 
prepared with the purpose of highlighting these policies. 

In All cASes. the policies that follow have been ststod in s format 
that maXes clear what is being reco»»endod for legislation and fcay 
not be a %M>rd-for-vord restatement of the original policy. 
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STATE-REPRESENTATIVE ASSESSMEHTS 

1« Frovldft ft genor*;! authority to conduct 8tato~reprees9ntstiv« 
aA»«s&m«nt8, with tho froquoncy and subjects dctdmined according 
to Board policy, vitb th* full coots borna by tho fedc^ral 
governmonn, and with Stato participation voluntary. 



tHiring 1990, both tho asscssoont cosisittee ot tho Coumril oZ Chiof 
Stato School Officers and tho National Forua on Educational 
statistics havo called for regular NAEP state-by-state assesssont. 
Tho nccNi f or t heso d.i I a has bi''en t*B»rhas i 2e4 in an ana 1 ys i o af 
Motional c^liK-tit ion goals r^c^^ntly proparod by i e Urban institute. 
Also, State NAEP saioplrs clearly are needed by tho Presidrnt and 
the Gcvor?)ara for their annual progroos r</port on Ai&«?rican 
oducat ion . 

Without testing on a corvnon in»trvi»i»nt under conaon renditions it 
would b« vi»ry difficult for citi^enA of any Stato to have Kcund 
cor^parative infonaation on ftow ttuch their students have learnc^d. 
SATs and ACTS are taken by self •'Selected groups. Coissercial 
otandardited testa have their "Lake woebegono effect" in which Boot 
children appear to be above average, stato criterion-reforenced 
tests vary widely. 

Congress authorised State NAEP on a trial basis in Sth grade vath 
in 19g0 and in 4th and Sth grade math plus 4th grade reading in 
1992, No subsequent state testing is authorised. These trial 
Stdto aasossments are being closely sonitored by the NAEP 
contractor and will be the subject or analysis across tho country. 
An independent evaluation, required by law, is being conducted by 
the National Acadosy of Education; reports ara due In lato 2991 and 
1993, 

If these trial assessments are successful and the infoni^ation 
useful 4 thfl support will likely be strong to procood with State* 
based aasessisents In the NAEP progras. if thar« are probless. 
changos in tha assesssents should bs made* The underlying reason 
for Stato assoAsasnts is perhaps stronger today than over and that 
is the need for sound, coaparfttivs data. 

The Board believes that full federal funding is needed to help aove 
to a testing progras in which NAEP date are collected for every 
State, that there is a clear public interest in collecting such 
data, and that it is the only way s national progress report on 
Aaarican education can be cosplete. 

Federal funding for sany years has supported the collection and 
reporting of education data by State. Those are essentially 
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liisited to ** input" data Includlfms •nrollit^nt, revonues, 
8xpcruSitur€», progrsB particip«tlon of studont«« pupil'^t«ach«r 
ratios, todcher ch*rftcteri»tlG», and high school 9ra<3uatlon 
requirement a. Thora ia no dif^fmta that a fodoral interest oKista 
for tha collection and reporting of auch infonsation at full caat 
to the government. 

However, soao analyata argue that there ia only a very lisited 
federal intrrofrt in collecting and reporting the outcomes of the 
application of the«e resources and reguiresenta by StatOf that tho 
outcoBQ of the education proceaa is a State concern and, therefore. 
State participation in HAEP should be livited to those voluntarily 
willing to aseusw part of the costa. 

Thi9 view is perplrxit^. Alma&t no valid informal inn rxists Ahout 
education outcoi&ea at lovela of analysis that bear on national and 
State policymaking. Policymakers at all levels are increasingly 
demanding such inforaation and only the federal governstent is 
positioned, through NACP* to provide It, Additionally, the asount 
of ntonoy Involved in not excessive, particularly in comparison to 
Qther federal education initiativ«>i?. 

The fact alono that President Dash and the Governors have, for the 
first time in our nation*a history, set national education goals 
for the United statea representa m soa change in the partnerahip 
in education botveon th* federal and state governnenta* Tha fact 
that those goala describe outcoBoa undaracorea tha concern for 
result a. Kesponaibtlity for education ia flrat of all « stata 
matter; and on thi» basis States will b« expected to do tha "heavy 
lifting** in effecting reforms and istproving education perforoance. 
It seems, therefore < fair and appropriate that the Federal 
government, in fulfilling ita role in the partnership, provide 
resources needed to help assess tha resulta of State education 
improvement efforts. 

Finally, while provisions related to voluntary costs to tha Statea 
should ba removed, those related to voluntary participation ia the 
aetscssment should not. The intent is to encourage full State 
participation in NAEP by removing a federally vrought impediment; 
under no circa&stanca should participation by a State In KAEP be 
mandated I conatttuta m requirement for receiving any federal 
benef it» or in any «ay be other than voluntery. 
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ANNUAL NAEP ASSESSHEHTS 

2. Provide for annual rather than biennial ftssrormRnts in thQ 
National Aftsessoent of Educational Pfoqtcss. 

It would be better to 9ivo MAEP on a rotating schedule of three 
fiubjcctB per year instead of Beintsinlng tlio current practice of 
tefttin^ three to five eubjects every tvo years. 

Annual NAEP testinc} and reporting «r© necessary to provide tiaely 
and fluff Iclent data for policy-Bakera and the r"J>Ii<?- This would 
NAEP to provide Xey lac^asures of acadeisic achievcwent for the 
annual progress reports on Ajnerican education, called for last y^ar 
at the Education Sumait between Prt'aident Bush and tho notion's 
Governors. 

For example, reading and mathematics plu» one other subject could 
be tested in oven-nuiobpred years. Science and tvo other subjects 
could be tested in odd-nunibered years. Annual schedules for data 
collection and reporting wou?d produce cost savings by stabilizing 
work-flow and eliainatlng separate pretest saoplee for new items. 

An annual KAEP would produce Httl© net increase In test burden 
because the current evory-other-year cycle includes up to five 
tests each testing year and extensive pretesting in off-years. 
Also, testing in three subjects annually would provide aople 
opportunity on a scheduled basis, perhaps every six years or so, 
to tost in such subject areas as foreign languages, the arts, and 
economics, which now could only t>e assessed sporadically • The 
Board already is considering those subjects for possible 
assessments in 1996. It is important to test a ran^e of subjects 
to discourage any narrowing of the curriculum. 

NAEP conducted annual testing fro* 1969 until 1980 when testing was 
placed on a biennial schedule to fyave funds, so this proposal is 
not without precedent. Amending the current authority to permit 
dfinual testing and reporting rather than mandating a biennial cycle 
will provide more flexibility in the conduct of KAEP and allow it 
to be Jboro responsive a» inforaation ne«ds evolve. Of course, the 
number of subject© actually assessed would be subject to the 
availability of appropriations. 
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BK.SIXWIWG LOCAL OPTlUfiS TO USE NAEP 

i. Eliainatc th« prohibit fon •gainst the uso of NAEP te»t itcas and 
i€i>otti»>q below tho State level by States, school di»trlct», «nd 
non-public schools and Lchool organ! Fat iona. Modify the NAEP 
er»nf ld«»nt lality provision to pt imlt States, srhool dic<tricts« and 
n»jn fHibl ic sirhools and school or^jan! z.»t ions to use NAEP for 
pri;vidin9 roauUs on individual districts, schools and f>tudc?ntA. 
The cost of such usos of NAEP and of rer- rtinq would be paid by the 
te<{uestin9 a^t-ncy- Reporting of inuivioual Btudent scor ca would 
continue to bo prohibited. 



Fat i9n^)« 

Th^'i^e » li.n..jps At to leJatod. They would r<-stOfo t^e local option to 
»i?;e tiM.V iiiat .states, localities, and svhnols pTrvloufsly had prior 
to 1988. The Gov<»inirKi Board has never N»i99i'sted--nor does it 
noW'-th»t the organisation which corolucts NAKP should erttablifih 
•.amplen in individual schools and school dii^trictR as part of the 
rf-f/tilAr NAKP c<.»tra*t and re|M>i t th<* rc'iults. Kuwrver, it is 
hi*}hiy flf'!iirrthlr that States, Kchool diNtritta, .^nd schools have 
th€» opt»on--if they wish to pay for it--of usinq NAF.P to measure 
their own s<:^ool9, as sooe have done in the past. 

Durinq the l«70s and 1980s a number of States used NAEP iteas and 
even NAKP teets to gather information on school and school district 
perfomanco. NAEP tests and test itess also had been us&d, apart 
f ro» the re^lar assesswent, for individual student testing in »any 
State and lo<:al testing prograim. All this activity was a 
desirable effort to provide useful data to students, parents, and 
schools. 

However, in 198S Congress enacted two provisions under NAEP that 
prevent State and local use of NAEP. The first prohibits any use 
below the State lovel of NAEP •test I teas and test data employed 
in the istatft NAEPJ pilot program." Although the statutory 
prohibition pertains only to questions used in the state-by-state 
dssessQpnts of ig^O and i992, NCES haa applied it to all NAEP 
exercises starting in 1990, Tho second provision reguires tf»© 
Couisa loner to ensure *that all personally Identifiable 
information about students, tltelr educational perforumsa, and 
their fast lies and that Information vith respact to Individual 
schools resaln confidential...* 

In July I99ft« th« National Forua on Educational Statistics, s group 
of SO State representativas and federal agency offlclala convened 
by NCES^ urged that States, "i' i*i»h, (should bs able to} 

analyse... student achleveMftt {on NABP| . . .so that covpari sons could 
bm sade aaoog edtic:ation imlts by significant subgroups." 

even though aui^rters say the prohibition and tho confidentiality 
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provision protc*ct IocaI intcx«>t>ta and family privacy, thoy also 
havo an opposite ef fwt--of cJ»nyli>g tha previously avsitobl* option 
of usinq NAKP If 1 ocelli tips vish to do so. The Board tx^lievo* that 
thcfse two provisions B^ol>ld be amende to clarify that they ai^ly 
to th« federal gov#rnBf»nt and its contractors in the conduct of 
NAKP; to p*»nBit !;tat& and Iwal use and au9m<^ntat Ion of HAtP at 
Imal option and vo«t *^nd pursuant to proir«»dures .ibl ifthed by the 
Coanissionpr to ensure trnt security » unifom adsinistrat ion and 
valid reporting; and to ptohibit ptiblic re|>ortin9 of individual 
s>t txdent t?cox es . 

X^rqc-acalB use of NAKP raises K«'rious issues of t«»Kt security and 
unifom ti'ot administration .«nd r€»jK>rtinq. If NAKP tCBts are 
{kUTi'haspd by rthools, ^iven vithoi*! C'Uj>orvision by thousands of 
fl.^'.f.f oofR t r.iilu'i s, ^ml i»ii'.<'tt for Hb^ny yr»iiii--As iH clone now with 
t or.ro»-rt- ial tc:it -t h* r<* is st»» i«>u« d.^m^#>r Icx al ii-suits vilX be 
inft,itrd and <.om|».»i l?f>ci«; with n.^tinn;*! NAKP nurm^ ronrf-nnd invalid. 
n«,us»uKe of d*f f»'r4*n4.*i»s in ©ot iv^t ion and po;;siblo propatat Ion there 
Wi>t>!d also be a serioi>*i problc-» in romparinq the re$»ults of 
studrnts vho t»w^k NAKf knowinq t hoy would q*»t individual scores 
with t tu>;:o in the* nat ional .ind State sawplos who i^re assured NAt,P 
<-.tn*t "lount" for th«m. Also, tho dosign of MAKP subject area 

.»'^r.f»f?sflit'nt- now divicii'd into bluOKs for eatrix sanpJ i n9*-would have 
to modified to piovide for individual student testing and 
repoi t i nq. 

These concerns, however ^ can be oddressod and should not be the 
b««sis for denying the utie of MAEP and NAKP itewa by States and 
local odurat ion *»nt it ios. 
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ZNTERMATZOHAL COMPMftlSOHS 

4. Provide Authority for § re9ular intematifmal co»pon«nt for 
HASP. 

EAtifiR&lfi 

HASP Ahould hBv« • r«9Ular int«m«tionaX compw)ent fimdod throu^ 
its rofular appropriation. Th« purpose of this coapoiHsnt vould 
no^ b# ass>gB>»nt beoftaony, 1.*. transforalng RABP into th# prlBary 
inatruMiit by ^ich participatii^ nations coapsrs tbsir 
psrfonuncs, Znstead, th« pur|H>s« of sn intsmat tonal coMponant 
in HAE? would ba to provlda inforwation to tialp Intarprat and 
understand NA£P results* just as State-representativa data ara 
intonded to cnhanca understanding of the performance of State 
education systems. 

The President and the GovenK»ra hive set national education goals 
in an international perspective (e.g. By the year 2000, ynitod 
States students will ba first in the world in aathematics and 
science) because a fuller understanding of the perfor^nce of U.S. 
students oust take into account the <K!ucation performance of our 
noighborSf trading partners and coa^titors. 

Making valid international coaparlsons is a vary coaplex 
undertaking. Curriculws and instructional practices* language and 
culture, student coapoaition, provlaiona for special education and 
disadvantaged students, rata of high achool coaplotlona« and the 
coapositlon of in-school eg* groups all vary vidsly fro* country 
to country and prevent easy coaparlsons. However. If ws ara to 
have sff active benchaarka for our own education practioas and 
performance^ they suat ba lnfors*d by expectations held for 
cosparabla student a In other countries and tha aggregate 
perforaance of those students. 

Current lav charges HhGB with "identifying af^rc^riata achieveaent 
goals for each age and grade in each aub^act area teatad under 
iNAEP|»" "developing standards for Interstate « regional and 
national coBparisons»* and *taking appropriate actions to Improve 
the for* and use of {NAEP}.* Faithfully fulfilling these 
obligations deaands not aaraly describing what 0.8. students know 
and can do, but dstarvining whether It la good enough. Halting this 
detarai nation ahould involve si^rt^lata analysia* In all anbiacta 
where practicable* of the perforaance of studenta in other nationa. 

For e^faaple, in setting achlavesent goala. It is aasential to 
assure that they are both challenging and reaaonabler exaainlng the 
ctkrrlcula of other coi»itrla« vould help in valcin? this 
dstarmlnatioii. Nhara otliar count r lea have coivarabla curricula in 
a particular aub^eet area and a comparable at-odant saapla oould ba 
drawn » it ^^Hild be useful to analyse respective pattema of atedant 
perforaance. Zt la not llltaly that thla wofuld involve nuseroua 
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count rlos for each asoeseiBent, ami it im not Intendod that such 
efforts would ftnount to a tru* international aasessaent. 

Information col 1m: ted under the International component voold to© 
directed solely at informing U*$. results. 

In 1988 an international comparison, u»ing NAEP iteaa in 
t&athom^tlcs and science, was conductsd by Educational Tast-ing 
Sarvice under grants fros tha National Science Foundation and tha 
V.Ss DeiMirtaent of Education, h siwilar cw^rativa aaaasamont^ 
vith siailar fuiKilng, ia planned by BTS in 1991. It will ttsa itmm 
developed jointly by participating count riaa, including aosa fro* 
NAEP. A loo planned are International asseasaenta in acianca and 
Bathenatica in 1994 and 1996 to be conducted by tha lEA^ the tising 
of which coincides vith the liKely NAE? schedule for theaa subjects 
and vhich way be linkcMJ with NAEP. While the Board applauds these 
efforts, tha fact is that they are conducted outside the regular 
NAEP fraaevork. Thus, they aro not a dependable source of 
information for doveloping policy (e.g., setting subject area 
achievement levels) and are not subject to NACB policy. 

With authority for an international cojnponent, HAEP could do a 
better and »ore complete job of reporting to tha American people 
on tha perfonsance of Aaer lean studenta. Haying epacific 
legialative authority for this purpoaa ia essential to avoid 
reliance on data intended for other purposes, to assure orderly 
and systenatic planning, preparation and data gathering, and to 
coordinate policy dovclopsent vith other agencies conducting 
international assessnenta. 
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ASSESS RANQB OF SKILLS 

5. ffodify thm NASP statoMni of purpoMi to clarify that aaseasaent 
will ccvor the broad ranga of skills and Xnovto^a (not just tha 
basic skills) in tha subject areas to bo assessad. 



Rationale 

This clarifying asendoent should bo non-controvaraial . It raflects 
the viav of sob^act satter and testing avpertac ai^rassad through 
ths consensus procaaa and In public haaringa wa hava hald» that 
NAEP assoBs»enta should includa an axasi nation of tha highar order 
thinking skills of U.S. students as wall aa *basic aXllls." Our 
policy and practice already incorpoirata this view, but we believe 
specifying it in the law would be usefvl. 
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REGIONAL SMIPLES 

6. Replsc« thm requirenent to oaploy reqionally-^ropreoontat ive 
sanplcft and produce regional ly-repr<?9entat I vd data vith Ian9ua9<» 
that would sake doing so a perBissive activity. 



R^ional reporting was fro« tho start of nAEP intended a« a 
suhstitute for Statd reporting. Froa a policy perspective, there 
is littlA cause for action that can bo dnrivod froo such data, and 
there is 1 ittlc evidence that such reporting has had any effect on 
education policy. 

With the possible Advent of ro^ulart predictable State level data 
collection, tho nc^d for drawing regional samples declines. While 
we do not advocate prohibiting regional sataples and reporting, we 
do envision a tiee in the future vhen such sampling and reporting 
may be unnecessary. Thus, the Board should have the flexibility 
to recomoend its application in the future Consistent with other 
( hinges in KAEP. 
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IHOBPENDraiCI or NAGB 

7. Elipinatv 5»biauitle« and conflicting provisions in the KAEP 
legislation regarding tho role and responsibilities of tho Kational 
As&esnent Governing Board C<*ACB} . 

The N/^P legislation gives t^ACB the authority to "fomiilate the 
policy guidelines for the National Assessment, •» to "exercise it* 
functions, powers and duties. . . independent of the Secretary and th« 
other offices awl officers of the Dopartsent of Education,- to 
"exercise its independent judgnent, free froa inappropriate 
influences and special intoresta," and to "hire its ovn fataff." 

TAken together, tnose provisions could not be sore clear in their 
intent to establish an iiwJependcnt policy body for MAEP insulated 
frOT the Jurisdiction of the Department of Education. However, 
other provisions reduce this clarity. 

Under the WAEP legislation, the Co*a»lssioner of Education 
Statistics carries out KABP "Iwjith the fldvAgg of the National 
Assesssent Governing Board [enqphasis added]. » Further, the law 
provide* that HiVSB "snail have the authorities authorised by tho 
Federal Advisory Conaiittee Act (FACA) and shall be subject to the 
open seeting provisions of that law." These provisions have been 
interpreted by the Departaent to Man that KACB is an advisory 
cowittee subject to tepartsent policies and procedures for 
advisory conittees and that »AOB policy decisions sust only b« 
given consideration by ths Secretary. Althou^ the Departsont 
now agrees that NAGB Is not an advisory coBUiittee, it continues to 
oversee NAGB activities as If It were, due in large part to the 
reference to FACA. Still unresolved is the degree to vhich the 
I>epart»ent is obliged to observe policy establl^ed for KAEP by 
NACB« rather than verely to consider It as advice. 

Another Issue relates to the NAGB budgst, vhi<di« as a set^aside 
within the RABP line itos In the DepartBent*s Kesearch. Statistics 
and improvement apprc^rlation, le in fact the Secretary's 
responsibility to adsinister. The contradiction iB painfully 
obvious— independence trtm the Departaent is incoi^tible with the 
i^ecretary^s bi^et responsibility. So far this issue has beon 
sai>aged in a spirit of cooperation, but it mains a potentially 
fatal flaw in )lAEP*s governance design. 

There is a corollary to the biKtget issue. tbm Dopartvent has 
Bssertad that the ftecretary** responsibility for adainlsterin? ^ 
■onitoring Departvent appr^riaticms brings KAGB activities UDdu 
it« jurisdiction. ttere cowXianos with gover»Mnt-wido 
roattireaeDts for fiscal accountability are at issue there is no 
dieafnment. but assortioRS hf the Dopartasnt that its fiscal 
authority under the Ottrrant arrangeaent extenda to review and 
claaranca of the intellectual activity of the Board and ita ataf f 
in conduoting ita work U InooHpatlbla vitb nOB^a atatotory 
aiaaion and is siaply unaee^ptabla. 

lO 
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AAolhttr part of tlm lav ^lovides that 'Th^ Secret* ry My appoint, 
at th» direction of tha Board* up to •ix technical •^loyaaa undar 
an axcepted service appointing authority. One* again, vhila 
mana9od in a spirit of cooperation, this provision haa tha 
potential for eroding NAGB indopendonca and contradicts th* 
provialon that tha Board shall hira its own staff. 

An araa of ai^lguity relates to evaluation of HASP* evaluation, 
particularly as prescribed undor tho naep legislation, is intended 
to bear on policy foraulation. h conaonly accepted standard for 
objectivity involves insulation of the ovaluator froa the subject 
of the evaluation. Tha HhBP legislation places responsibility for 
evaluation of NAEP with the ComnisBionsr and is silent about the 
evaluation responsibility of NAGB. We believe that this is tha 
opposite of sound practice for assuring objective evaluations and 
would advocate for an dwendro«>nt that would add to NAGB's functions 
the formilation of policy for tho conduct of evaluations of HAEP. 

While the lav is very specific regarding the balance of 
responsibilitiea between the Secretary and tho Board in making 
appointments to the Board, it is silent with respect to how NAGS 
will organ ice itself in per forming its functions. The Board 
believes thet the independencs provisions in the law authorize 
total autonos" with respect to its internal workings. The 
Department believes thie to be true with one except ion* -the 
appointsent of the Board Chainwn by the Secretary. Although this 
issue tu ^ been managed .n a spirit of good vil« and cooperation, 
it resains a sajor point of contention. 

These issues are fundasental. Their existence erodes the 
principles on which NACB was established: independence fr^ the 
Department of Education, tripartite check and balance NAEP 
governance structure, and freedom froa Inappropriate influences and 
special interests. 

The goal that the Board seekff is no Bore than what Congress 
intended — a truly indei^r^nt body whose policies direct the 
conduct of KAEP. The NAEP legislation does not define 
independence, but the Board believes that it includes three 
factors f (a) freedo* fro* Inapprc^riate Influencos and special 
interests in Mking adBinistrstivs and policy decisions^ (b) 
assurancs that policy decisions made by HAGB will be isplesented 
in the eonduot of HABPi and <c) full, essy and tiaely access to 
information that NACB needs fron a variety of sources, including 
the Departsent, thtr Congress, education practitioners, and the NABP 
contractor > 

Legislation designed to a<^levs these ends could tske jsany forsa^ 
fros sodifying or eliminating sose of ths provisions cited above 
to establishing HAGB with its current authorities as an indsperaSent 
agency outside of the D^rtwent. Should the Depertaent of 
education be conteiq>lating changes in kaGB's authority as part of 
its reauthorieation proposal, tha Board would look forward to 
diacuMing this Mttor. 
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NACB Nf9«6ER5HIP 

8. Provide for th© continuing mc»inb«rship of th« prcBont Board 
according to current tonsm and for continuing the current 
provisions for fiitir>g vacancies. Add a provision that oxplicitly 
lifAits Bo«»rd mcwbors to two consecutive trrms of up to four ycArs 
in each ter». Encourage the Secretary to consider a Candtd^to'tt 
previous experience on the Board And the ov<»r^ll evperience of the 
Board when making Appointment a. 



The current legif?lation containt* piovisions for the transition in 
90vetn.%mrc from the Asse^JsPeot ToJ i«.:y Committee to NAGB. Thef;c 
provisions are now inappl ir.ihle and will l>e rcnovrd whcm'vcr NAKP 
is riMuthori n*»w pioviaions should U>*tve no doubt «*bout the 
cont inUi^t ion of the current member ship. 

Doring one thirteen month poriod in l989-*>0, X4of the 23 appointed 
r»h«»r» of tho Btv^tJ w£»rt? rt»placrd. This turnover rate is too high 
to assure the btabi X ity .^nd cont inuit y th,»t w3S envisioned in 
providing for terns of up to four years. Providing for two tenao 
and taking Board experience into account in Baking appointments 
will help assure stability. 



%2 



ERLC 



V 



31 




National Assessment Gov erning Board 



NATIONAL ASSESSHUrr COVLAHINC BOARD 

Positions on 
Ths ruturs of ths Hstionsl As»«sS0«nt 



APPROVED: Deceab«r9, 1919 
AtKeetln9 InAusttnt TtxftS 



Ttst Nstionsi Assessment CQVfirnin9 Board< Bindful of itf 
statutory responsibility to $eeX to Isprovs ths Natlonsl Assoss&ent 
of EaucAtional Pro^resft, hereby adopts thcss positions and 
rscom«iHlst lonst 

1. Ths yatlonsl Assessnent of gducat^enyl Prgg ysss shpul^ 
pr^trfii tnforftatton for an annual report card tsstlfto at \mAm^ 
three sub jects each ysar* 

The 6xact conf IgMration should b« dstsninsa sftsr s esrsfal 
analysis of Cost and Asna^csent cons idarat ions Involvsd in such «n 
incroass ovsr ths curront svaryothor-yesr tsstln9 cycls. 

Ai>nual HA£P tsstin9 and rsportln^ srs nscsssary to provids 
ti&ely arvd sufficient data for policy saksrs and ths public. Ondsr 
its currant schsduls, hovsvsr, ths National Asssssasnt is vosfslly 
incomplsts. ftoporting assess&snt dat* each ysar vill alloy HASP to 
b«coBS ths ksy »oasurs of acadSBic achlsvessnt in ths annual Baport 
Cards on Assrican school s# called for in SeptSBbsr at tha sduoation 
suii»lt botvesn Prssidsnt Bush and ths nation** governors* Also, 
regular annual data ars nosdsd froA HASP to rsplacs ths education 
Departsent's annual "vail chart"^ vhieh rsliss on 8AT and ACT 
scorss* 
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gtrti '-^ P^^^Q" in all ijnitcti and al l thrf aratfy ImU Uth^pth. 

tt^rticipatftr tiQvrnnftnt ahtmld aav tha fMl gf tht 

tn 1990 sach Stat* vuat pay about $100«000 to ta1(« in 
th« Btate*by-stat« »4thcaatica ass«aasaftt. Sevaral hava eit«d this 
axpansa as a reason for not f^articlpatin^. 

pi^»ipnala 

Tha Governing Board vishca to hava a fuH tf9tin9 pro^ras in 
vhich HMP <3ata arc collected for every atata. Thara ia a clear 
public interest in obtainin9 such data; it is the only vay a 
national l^^s^ort Card can be complete. The Doard believes it would 
be inadvisable to r<7quira states to participate in HhB?, Rovever, 
full federal funding vovild sake it lesa liXely that statea vould 
decline to take part. By having }thtP testing in all states, th« 
Board also wishes to end duplicate national aanplaa ae aoon as 
feasible. 



3. IHf ^ft3£.e£airt9 — vrqea Connress to remove tha 

pyot^ ^bitich n T?si»«t the use of NA£P tests and dat^ y<^p9rt^^y>ff tmlW 
thff ff»^<*ta level. 

publication of individual student scores should continue to 
b« prohibited. Any additional coste for testing belov the etatS" 
level sasples should be paid by participating states or localities. 

ROtigMlt 

The clause to be eli»inated--Subsection <4)<C) of P.Xm 100- 
297-- reads aa follows: *The use of National As&esssent test it ess 
and test data esployad in the pilot prograa authorised in 
eubsection (2) (C) to rank, comparoi or othsrvise evsluatt 
Indl' idual studenta, echools, or school districts ia pr^ibited«* 

This ch«r9^ i^»ld perait KAEP teet iteae and data to be uaed 
at: the levels— school district end a<0»ool buildliig—vhers aanv 
i sport ant educational dacieione ere aada. It would provide helpful 
information to parents and othera intereated in school iaproveaent« 
ey aaking NA£P aore useful to local schools^ offering this <»tion 
would encourage thea to participate in national and state aai^ling. 
Statea and localities would be abla to tia their ova tegular 
taating progress into liABP. Before the pr<^ibition was anacted in 
1^$$^ aany schools and dietricta regueeted KAEP r^rts on students 
who participated in national saaplingi several etates adainietared 
MAtP itaae. 
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cth«r nitl<m» ^tould b* Invittd to pmiGlp«t« in nhMf m • 
rMilftr »chidiiU. MASP t#flt« 9hpmli ^ •d»liii9t«r«a te 
r«pr«s«nt«tiv« MBpl»v Qt th»ir •tvdvntt^ vMcfi Bight Mrva 
r«r«r«nctt point* for *chl«v9»«nt in A»«rle»n •choolft. 

Th« Report Card for A9«rie«n atttdtnta !• incovpUtc if it 4o«« 
not includ* • regular b«ro»«t«r of AC«d»mic ftcm^vratnt la ftft 
int«rnatioft«i cont«Jft, Today th»ro i» no tu^ tAroMtor. 



5. llit_g9V^^^ ^»<' B oard l K»ligv tho rcleasd of WAgP dat* Mt 
bfl-gr^f-I^J^ ieau<?Bt» If otaff to report h)t Jftrch on Alt 
tttrrgJfT^^^ltf h# takftn to redoco tho ti— for tgportlnq BMP tttt 
rtgj^ita/ i ncmtfinq Bog>iblt tf eaign cbftfwm ir i»etfii. 

Xt h*« Uktn 21 to 24 »ontti« aftor toctlng to roport litfP 
retiult*. TM» 1» »ch«dul«4 to l«prov« to 15 to ii sontho for tbo 
1990 »ssessBtnt. But HAEP vlll otiU bo oucli alovor thmn tha 
Collooo Board and Asoricon Col logo Totting ProfroB Mwory roporto 
•nd alBoot tU itato tooting prograno, vhlcH ustttlly Mko dito 
public in t^*n alx oontKo. 

Rttiomilo 

Tho long^aUy in roporting oovtraly roducos KASP^o uaofulAoM 
and iapoct. It vill bo difficult for KABP to oorvo ao on anmial 
Roport Cord if it la alvaya soro than a yaar lata. 
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carefully^ Including any trada-^f fa that oigbt ba nacoaaary If KAtP 
daoign, taat coottructlon, and tooting practicoo aro changad* Iba 
ot«4y vill ioclttda a coat-bonaf it analyala of any racrasandatim. 
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includ«9 th« concensus proc«os tor goal* AAd o&]octivo»« 
preparation of to»t ti^clf Icationo and qutvtfono, rovlov of 
quootioAA by NAOe, ar^ r«vieva of co9nitivo Ua»a and background 
quostionfi by th» Off ica of Kanagosont and Budgat and tho Education 
D«part»«nt* 

Bflt jonala 

Tha fttudy ahould includa ell aapocta of tha ttat praparatlon 
cycla. Special ^vasination ahould b« sftda of th» ia|^ct OA HASP of 
thm Paporvork Reduction hc% pf 19«0i ravi«v« b> 0N9 and ttia 
Education CVep^rti^nt prascntly taka a total of aight oontha par 
cycla, Tha atvdy vill inciuda a coat*banafit analyaia of any 
rocoB0esv5st lonf • 
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Attention to highar-leval akilla and knovladfo aa vail oa 
basic akilla i» claarly in Una vlth afforta to atrangtiMn tha 
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S«»pU sit** should b* •nl«ir9*4 to provid* InfonMtlon on 
9r<mpa much m Idv-tncov* childron tnd tho«« sttondinQ prlv«to 
•chooU. 

Thi» would provide iBportftnt inforMtton for •duc«tor« and 
policy BAXcrs and for *n»l)r»ift of schools. TTis dofinltion of low- 
IncoM studonts My bt difficult, bn-, KAEP could thoss who 

qvslify for pArticlptt&on in f«dsrf« frst or roducod-prict lunch 
pro9r&ss or childrsn vhp sttond Ch^ptsr X schools* 



Th« roltt and r«»ponsibilities 9f ths Governing Bosrd should 
b# cUrified by new lo^isUtioh. 

Bit Unftlt 

mm currsnt •ystoB of dividsd suthorlty My issd to confusion 
Sf>d disputes which would ha»por tnm HASP pro9rss. Covornancs and 
sd»inistrativo aupotvision of ths national Aasosasant ahould ba 
rational trod and focuasod, Ovarlap and duplication of furctions of 
HMB and NCES Staffs should bs sliBlnatad* 
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Chairman Kildee. Thank you very much. We will go through the 
panel then address our questions singularly or jointly at the con- 
clusion. 

Dr. KoRETZ. Mr. Chairman^ thank you for the opportunity to ad- 
dress the subcommittee atout the National Assessment and about 
national testing i^ues in general. 

I also mil aboreviate mv remarks and ask that my full t^timony 
be submitted to the record. 

Chairman Kildee. Without objection, it will be included. 

Dr. KoRETZ. I am going to start with a couple of general issues 
about national testing bmiuse those are the underpinnings for the 
specific comments I will make about the National Assessment. 

I think to judge how well the National Ass^ment and other 
tests can meet the rapidly rising expectations people in the policy 
world have for them, it is necassary to bear three issues in mind. 

The first is the multiple functions that these tests are being 
asked to serve. The second is the problem of corruptibility of test 
scores, and the third is the unavoidable technical complexity of the 
whole enterprise. 

Current propt^ls ask the NAEP to do three things: to monitor 
achievement which is its traditional function, to describe what our 
kids can do; second, to evaluate educational prc^rams or policies; 
and third, I think underneath it all to hold [^ple accountable for 
performance. 

The pell mell rush to increase testing that the letter that Mr. 
Goodling read referred to is not a pell mell rush for more monitor- 
ing. It is a pell mell rush to judgment. It is, I think, currently re- 
flective of a desire for more evaluation and more accountability. 

Unfortunately, it is very difficult, some would say impo^ible, for 
a single test to serve all three of those functions at the same time. 
There are two reasons for this. I can come back to them in more 
detail later if you would like or in writing. The first is that tests 
have to be designed differently for those different functions, and 
the second, which I think is perhaps more imix>rtant, is that using 
a test for accountability can undermine its accuracy as a monitor- 
ing tool, as a source of gotwl, descriptive information about what 
our kids can do. 

And this points to the second of my general issues which is the 
corruptibility of tests. In the policy world there is a widespread 
view that tests are kind of like a fever thermometer, a measure 
that you can always trust. If a fever thermometer shows a fever 
going down, you can be confident that the fever has gone down. I 
don't think that is necessarily the case, and, in fact, we are sur- 
rounded by measures that are not so incorruptible. 

One example might be airline on-time statistics. If you suddenly 
find that you are getting someplace on time, is it really because 
performance has improv^, or is it because the formal length of the 
flight has been increased to allow for the same delays on the 
tarmac? If it is the latter, which I think in some cases it has been, 
it do^n*t t^ll you anything about how long it gets to where you 
want to go. 

In the sarne way tests can be corrupted if they are used for ac- 
countability. Scores can go up, but actual achievement, the time it 
takes to get to Detroit, may not change a whit, and this is the 
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basis, one basis, for what we now call the Lake Woeb^one phe- 
nomenon, the puzzling facts that most States and districts claim to 
be above average. 

One reason this can happen is very simple, and it was also re- 
ferred to in the letter that Mr. Goodling read, which is teaching to 
the test. Teaching to the test focuses instruction on the narrow con- 
tent of the test rather than the much breeder range of achieve- 
ment that the test really is supposed to Liform us about. 

The final general point is that large scale assessments such as 
N AEP, if they are to provide useful information that we can trust, 
are technically highly complex. Currently there is a very wide- 
spread impatience in the policy community with technical aspects 
of measurement. They are seen in many quarters as kind of an irri- 
tating obstacle to changes that people want to make, but that im- 
patience doesn't in any way decrease their importance. 

With those basic points in mind, let me turn to the NAEP itself, 
the National Assessment. 

In my view the National Assessment remains ideally suited to its 
original function of monitoring what our kids can do. Moreover^ 
there is at present nothing else that we have that is a clooe substi- 
tute for it. Nothing else tests nationally representative samples of 
kids at frequent intervals on a wide variety of subjects, and in addi- 
tion the NAEP, unlike many State level tests, is not yet taught to 
in a way that gives us reason to distrust its results. 

So if we lose the NAEP s ability to monitor achievement, we 
have lost a lot 

The National Assessment could be improved in this regard. It is 
not to say it is perfect for monitoring. Some of the potential im- 
provements, however, would cost money, and money is scarce not 
only because of the general budgetary situation but bwause a lot of 
money is going into assessment goals that I think are quite futile 
such as using State comparisons to evaluate State educational pro- 
grams. 

I won t take time to go into what some of the improvements 
might be, but I will just mention one. The Congress has been inter- 
ested for at least eight or nine years in the performance of high 
achieving students in the pipeline, future scientists and mathema- 
ticians. The National Assessment is not designed well to assess the 
performance of those students. We don't have a trustworthy indica- 
tor of the performance of thcwe students. We could make the Na- 
tional Assessment into one if we wanted to spend the money to do 
it. 

The second function, evaluation, is really the key to the argu- 
ment about State comparisons using NAEP. If you read the justifi- 
cations for State comparisons, it seems to be very clear nobody is 
really very interested in simple, descriptive infornxation that sa)rs 
that mathematics achievement is not as good in Mississippi as it is 
in Minnesota, 

What people want, and Dr. Walberg alluded to this, is some indi- 
cation of which States have better educational pn«rams or poli- 
cies. The flv in the ointment is that there is simply no way that the 
National Assessment as it is currently structured can provide 
meaningful information of that sort. When one State outscores an- 



38 

other, it might be for educational reasons or it might not, and we 
won't know when it is. 

And moreover when State A does outscore State B for education- 
al rea^nst we won't able to tell which educational reasons. 

The same ambiguities and uncertainties would arise if the Na- 
tional Assessment were used at the la^al level if the statute were 
changed to allow that. 

The reasons for this are too complex to discuss in detail here. I 
would be happy to come back to them, but they really boil down to 
one simple point which is to say that a policy causes educational 
achievement to go up or down, you have to be able to rule out rea- 
sonable alternative explanations of the sort that, Mr. Goodling« you 
mentioned in your letter. 

The National Assessment is l ot designed to do that. It doesn't 
track the growth of students over time. It includes only very weak 
information on the background and characteristics of students or 
their communities, and it doesn't include much information on the 
educational policies or practices in the districts where students go 
to school. 

Some people say that is all right. It is stil! adequate for some 
degree of evaluation of policies and programs, but in saying that, 
they are implicitly accepting a standard of evidence that is vastly 
looser than we would be willing to accept in any other domain. We 
wouldn*t, for instance, in evaluating health programs or medical 
treatments, be willing to accept evaluations based on evident v that 
is flimsy. If we did, what we would have is incorrect evaluations 
that would sometimes mean that effective medical treatments go 
down the tubes and would sometimes mean that ineffective medical 
treatments would continue. The same will happen in education if 
we accept a low standard of evidence. 

The National Assessment could be redesigned to provide this 
function. There is no reason why it couldn't. It would require a 
whol^le redesign of the pn^ram. It would cost a lot of money, 
and al^nt a lot of additional m'^ney it would therefore take away 
from the things that are currently appropriately spent to increase 
the quality of the descriptive information that the National Assess- 
ment is designed to give us. 

The third functioHt wiiich is really very tidily tied to the second, 
is accountability. Using the National Assessment or another na- 
tional test for accountability purposes would in part run into the 
same difficulties I have just described because it isn't really very 
reasonable to hold people accountable for differences in educational 
F>erformance unless you can really say that they are responsible for 
them. 

But there is another reason to be worried about accountability 
beyond that which is that using the National Assessment in that 
way could undermine its value as our only high quality nationally 
representative indicator of performance. 

The issue, as I suggested earlier, is teaching to the test. In the 
educational community and in the policy community there is a lot 
of disagreement about whether teaching to the test is, from an in- 
structional point of view, desirable or pernicious. I think for 
present purposes we don't have to be distracted by that argument. 
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I think, by the way, it is largely pernicious, but that is l^ide the 
point . . 

The point is regardless of whether it is desirable or pernicious, it 
can wreak havoc with the information, the descriptive information, 
we want from a monitoring sj^tem. 

The National Assrasment does include some protections against 
that kind of corruption. One is fairly rigorous test security. The 
second, which Dr. Walberg mentioned, is a sampling system where 
each kid takes only a fraction of the test. It is by no means clear to 
me that those are enough to protect it from corruption. 

One last point on this is that I think local comparisons are by 
and large riskier than State comparisons in this regard because 
they increase both the incentive to raise scores and the tools which 
people can raise scores. 

ProfKjnents of local comparison say, "Exactly. That is what we 
want people to do. We want them to increase their performance " 
Regardless of whether that is a desirable thing from an educational 
point of view, it is not a desirable thing if you want the National 
Assessment to tell you what kids can do. 

There is a solution to this dilemma which is to use the National 
Assessment for monitoring and to uro something else for account- 
ability if test based accountability is what you want to do. 

Finally, I would just like to make a couple of comments about 
the National Assessment Governing Board in the context of my 
third point which is the unavoidable technical complexity of the 
National Asse^ment. 

Under the current statute the National Assessment Governing 
Board has an extremely broad purview. It includes policy questions 
that don't require a great deal of technical expertise and that prob- 
ably should not be considered the province of technical experts. 

It also, however, includes explicitly in the statute authority over 
very technically complex issues such as designing the methodology 
of the assessment. 

The National A^essment Governing Board is simply not consti- 
tuted appropriately for the second category of functions. Right now 
only two of the 23 seats on the committee, on the board, are set 
aside for experts in testing and measurement, unlike, say, the Edu- 
cational Testing Service, which is the prime contractor for the 
NAEP, does not hive a standing panel of experts to advise it. 

The contrast with other government funded ^ncies or organiza- 
tions that are responsible for making technical decisions, an exam- 
ple I would use is National Institutes of Health or ADAMHA 
review panels. It is very striking. 

We would not constitute a panel under NIH s purview to design 
clinical trials that comprise primarily people who don't know how 
to design clinical trials; but that is precisely what we have done 
with the National Assessment Governing Board. 
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This lack of expertise is important for several reasons. One is, as 
1 suggest^ before, there really is no way to separate technical 
from substantive considerations in testing. In medicine technical 
considerations tell you whether a medicine works. In testing tech- 
nical considerations tell you whether you can t^lieve the informa- 
tion you get or alternatively what it means. 

Second, there is extraorainary political pressure now from vari- 
ous education constituencies to chan^ the a^essment, to use it in 
many new waj^ and to get moving with th^e changes. The Nation- 
al Asse^ment Governing Board needs the expertise to understand 
when those change are practical and when not and to decide what 
they have to do to accommodate th<se pressures in a technically 
defensible manner. 

Therefore, I think the efTectivene^ of the National Assessment 
could be improved by changing the statute to insure reasonable 
representation of technical expertise on the governing board. How 
much change is required, I think, depends on whether or not the 
purview of the board remains as it is. 

If the board does retain authority over highly complex issu^, I 
would suggest that it might be reasonable to increase the represen- 
tation of technical experts to as much as a third of the board which 
would be eight members, currently. 

If authority over some entirely technical issues were removed 
from the board's purview or made joint purview of the board and 
some other agency, perhaps a more mcxiest change would make 
sense such as increasing the representation to four. 

In either case I think a requirement should be put in place that 
the individuals chosen for those slots are widely recognized as ex- 
perts by their professional peers throughout the United States. 
There is currently no such requirement. Again, I think that NIH 
and ADAMHA review panels might provide us with some hints 
about how we could assure that in the future. 

Finally, to wrap up since I have gone on a bit to long, clearly we 
are at a turning point, not just for the National Assessment but for 
educational assessment and assessment at the national level in 
general. The National Asse^ment has already, in my view, taken 
serious States in its trial State assessment away from its tradition- 
al monitoring function and toward the functions of evaluation and 
accountability, and I think many of the profx^ls on the table now, 
starting with the last session, would push it further in that direc- 
tion. 

My own view is that the changes made to date were undesirable 
and that further changes in those directions would be ill advised. 
Absent dramatic changes in the structure of the NAEP, efforts to 
use it for evaluation are futile and pK)tentially misleading, and ef- 
forts to use it for accountability will simply undermine its value as 
our only sound indicator of the performance of our nation s youth. 

You may well decide that the monitoring function is not, in your 
view, the most import.ant function for the National Assessment. 
There are certainly many people who don't believe that it is the 
most important function. But as you deliberate that, I would ask 
you to bear in mind that although you will hear claims to the con- 
trary, we really can't have our cake and eat it, too. We can't have 
everything from the National Assessment. It either is going to do 
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what it was designed to do or it is going tc do something different, 
and we have to make a choice. My hct^ is that we will choose to 
continue using it and improving it as a tool for monitoring per- 
formance. 

Mr. Giairman, thank you again for the chance to address the 
subcommittee. I would be happy to take questions at any time. 
[The prepared statement of Daniel M. Koretz follows:] 



« 1 (I 

ERIC 



42 



STATEMENT OF 

Daniel M. Koretz 
Senior S<Knal Scientist 
The RAND Corporation' 

before the 

Subcommittee on Etemetitai /, Secondary, 
end Vocational Education 
Committee on Education and Labor 
U.S. House Representatives 

March 13.1991 



^ The views expressed here are solely those of the author. They do not 
necessarily represent the position of the RAND Corporation or its 
clients and we e not prepared in fulfillment of RAND*s contracts or 
grants. 



43 



Mr, Chairman, thank you for the opiwrtunity to address the 
Subcommittee about the National Aflscssment of Educational Progress 
<NAEP). Before turning specifically to the operation of and statutory basis 
for the NAEP, I will briefly address several general issues about testing 
that underlie ray specific recommendations for the NAEP. 

BACKGROUND 

The NAEP currently has an importance that would have been hard to 
c^nvision even a few years ago. The ferment about educational performance 
that began more than a decade ago continues to grow, and each wave of 
reform proposals further increases the importance accorded to testing. 
These trends are now expressed at the national level both in proposals for 
the NAEP and in proposals outside the federal government for the 
development of additional national tests that would be used-\mlike the 
NAEP"to test all students. 

Tr judge how well the NAEP and other tests can meet the rising 
expectations poUcymakers hold for them, it is necessary to consider three 
basic issues: the multiple functions that the tests are being asked to serve, 
the problem of corruptibility, and the unavoidable technical complexity of 
large-scale assessmtants. 

Currently, tests are being a-,'^ wd to serve many purposes, three of 
which are central to proposals for large-scale assessments such as the 
NAEP. These are: 

o Monitoring: providing a description of achievement. 

o Evaluating educational programs or poUdes: not just 
describing what students know, but also determining why 
they know it. 

o Holding peofde (students, teachers, administrators, or those 
holding political office) accountable for achievement. 

Monitoring is the traditional function of the NAEP; it was designed to 
inform debate and improve education by providing the nation's best 
description of the achievement of its youth. Traditionally, the NAEP had no 
evaluatiw or acoHiniability functions, but recent changes and proposals, 
such as the ncovement toward state comparisons, rest in large part on 
proponents' desires for a national test that can be used to ev«Ouate 
programs and policies and hold to hold people accountable fbr them. 
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Unfortunately, it is very difficult for one test, no matter how good, to 
serve all three fiincUons simultaneously. First, the different functions call 
for different designs. For example, for purpose s of monitoring, it is 
acceptable to sample students at a single point m their schooling and to usp 
resources primarily to insure adequately broad sampling of both students 
and tasks. To evaluate programs, howeve/, it is necessary to channel 
resources diflerentiy, to obtain more detailed data on factors that influence 
test scores and even t*- track the growth of individual students over time. 
Second, using a test for accountability can undermine ita accuracy, thus 
undermining its usefulness for monitoring. 

This points to the second isf.ue: the corruptibility of testa. Many 
people incorrectly view tests as an incorruptible measure, like a fever 
thermometer. If a fever thermometer shows a drop in temperature, one can 
be pretty sure that the fe^er itself has declined Many other common 
measures, however, are more easily undermined. On example might be 
airline on- time statistics. When an airline's on-time statistics improve, you 
might well ask why. Has the airline in fact decreased delays, or has it 
simply increased the estiniated time of its flights, to allow for long delays on 
the tarmac while remaining "on time?' If the latter, nothing has really 
happened to how long it takes to get to Detroit 

Similarly, tests can be corrupted; particularly when they are used for 
accountability, scores can 2n^,i^a8e more than actual achievement. This is 
one basis for the now well-known "Lake Wobegon " phenomenon: the fact 
that most states and districts now report themselves to be "above average." 
One reason this can happen is simple: teaching to the test. Tests are 
necessarily a very small sample of what we want stt'dents to know. We 
can't test their knowledge of tens of thousands of words, so we test them on 
a sample of 40 or 60. As long as the 60 are a reasonable sample and no one 
directs teaching al those particular words, this works: one can infer the 
level of mastery of thousands of words from mastery of the 60. What if, 
however, a teacher simply has students memorize tJie 60 words? Scores 
Rkyrocket, but total vocabulary is scarcely affected. This is a particularly 
extreme example, but less extreme teaching to the test can also inflate 
scores on the test significantly. 

The anal general point that must be borne in mind is that large-scale 
assessments such as NAEP-*if they are to provide us with useful, valid, and 
reliable information-are necessarily highly complex. Designing and 
implementing them requires substantial technical expertise in measurement 
aji J sampling. Even issues that seem on their face to be largely substantive 
rather than technical can have complex technical aspects that must be 
addressed if the test is to provide meaningful information. Currently, there 
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is widespread impatience with the technical aspects of measurement in 
much of the education policy community, but that impatience does not 
lessen the impact that technical considerations have on the information we 
obtain. 

With these basic points in mind, I would like to discuss the operation 
and statutory basis of the NAEP. 

MOMTORING 

The NAEP remains ideally suited to its traditional role of monitoring 
the achievement of the nation*8 youth. Moreover, its value for this purpose 
is hard to overstate, because nothing else is presently a close substitute for 
it. We have no other assessment that test* nationally representative 
samples of students at frequent intervals on a broad range of subject matter 
that we believe important. In addition, unlike many state-level tests, the 
NAEP ia not yet taught to in any way that would undermine ita results- 

Nonetheless, the NAEP could be improved to better serve this 
monitoring function. Some of the potential improvements wo\ild require 
resources that are currently very scarce, both because of the general 
budgetary situation and because the NAEFs resources are being siphoned 
into other uses, such as the futile eflbrt to evaluate states' educational 
programs. 

The improvements that might be made to the NAEP to monitor 
achievement better are many, but these are a few possibiHties: 

o Adding periodic supplements to assess high-achieving 
students, es|^al)y in mathematics and science. This 
jfToup is a mcgor focus of policy debate, but the NAEP, 
designed to assess the student population as a whole, lacks 
both the appropriate sample of students and the 
appropriate mix of tasks to assess this group well. 

o Augmenting the sampling of other groups of importance to 
policymakers, such as immigrant students and college- 
bound minority youth. 

o Strengthening the currently very Hmited array of 

background information collected to describe students and 
schools. 
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o Further exfmnding of the range of knowledge and tasks 
represented on the test. 

The last of these possible improvements points to a serious deficiency 
in the current administration of the NA£P: the insuflident weight given to 
important technical considerations in making certain decisions about the 
operation of the NAEP. The NAEP is moving rapidly to incorporate new 
forms of assessment ( particularly » various types of "performance 
assessment") and to decrease its reliance on multiple-choice questions. 
While this effort is laudable, it is difRcult; we don't know as much about 
alternative forms of assessment, and the NAEP has to invent things as it 
goes along. The difficulties inherent in making this change have not been 
taken into account adequately in charting the NAEP s operation. 

Currently, the Educational Testing Service (ETS), which is the prime 
contractor for the NAEP, is neither expected to evaluate these new forms of 
assessment adequately nor provided resources to do so. Rather, ETS is 
pressured to incorporate new assessment methods into the NAEP in the 
shortest possible time, even in the absence of any confirmation that they are 
of sufRciently high quality to be informative. At a recent meeting of a 
NAEP technical advisory committee, Artihie LaPointe, director of the NAEP, 
drew the accurate analogy of trying to retool the engine of a car while it is 
moving at 60 miles per hour. 

This specific problem could be addressed by changes in the authorizing 
statute. For example, the statute could provide for a period of time-- three 
years was a recent suggestion by some technical advisors to ETS--for new 
assessment methods to be developed and evaluated before they are 
incorporated into the NAEP for purposes of national reporting. Similarly, 
the statute could direct NCES to provide a reasonable set-aside of NAEP 
funds to be used for this purpose. (While there is room for debate on this 
point, it might be most efficient to divide these funds between the NAEP 
prime contractor and other independent research groups.) These changes 
would slow the adoption of new forms of assessment but would assure that 
those finally used are worthwhile. The more general problem of insufficient 
consideration of technical issues can also be addressed by statutory changes; 
I will discuss those shortly. 

Ironically, an additional way to improve the monitoring function of the 
NAEP would be to increase fimding for competing assessments. There are 
two reasons for this. One is the competition per se, which might over time 
lead to more effective or efficient ways of conducting the NAEP. The second 
reason, however, is the one I want to stress: fielding additional assessments 
wilt enable us to interpret the results of the NAEP with more clarity and 
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certainty. Because tests are only samples of what we want to assess and 
are technically complex, even fundamental resulta can difTer substantiaUy 
from one test to another. Put simply, it is risky to put too many eggs into 
any one testing basket, and even the NAEP can occasionally mislead us. 
The trustworthiness of the information provided by the NAEP therefore 
could be bolstered by increasing the amount of information from alternative 
sources. Clearly, additional assessments of the scope and frequency of the 
NAEP are not practical, but they are not needed for the purpose of 
validating the NAEP. The information needed might be obtained, for 
example, by funding periodic special studies and by augmenting the testmg 
associated with the fiducation Department's periodic longitudinal surveys. 

EVALUATION 

The current controversy about using NAEP for state comparisons 
hinges in large part on the reasonableness of u^n^ NAEP to evaluate 
policies and programs. State-level moni/orintf-providing descriptive 
information about the severity of educational problems-could be useful, but 
in fact few proponents of NAEP state comparisons appear to want that. 
What they want is information about the relative effectiveness of states* 
educational programs. 

The NAEP, however, as it is currently designed, cannot provide 
reliable information about the relative effectiveness of state programs. It can 
tell us which states have higher achievement, but not why they do. In some 
cases, State A will outscore State B because of better educational policies, 
but in other cases, the cause will be something else entirely, and we will 
rarely know which is which. In addition, when State A dees )utscore State 
B for educational reasons, we wilt oflen not know which eoacational factors 
are the cause. Precisely the same uncertainties would arise at the local 
level if the current statutory prohibition against using NAEP to evaluate or 
compare schools and districts were lifled. 

The reasons for this a> .00 complex to discuss in detail here, but they 
boil down to one essential point: the NAEP does not provide the information 
we would need to rule out reasonable alternative explanations of differences 
in scores or to pinpoint educational factors that might be responsible. 
Because the resources of the NAEP are channeled into providing the best 
affordable estimate of achievement, they are not channeled into producing 
other information needed for evaluation^ such as detailed information on 
family and community back^ound factors, educational policies and 
practices, and students* prior levels of achievement. 
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I have been in debates in which the proponents of NAEP etate 
comparisons insist that the NAEP is ackquate for drawing inferences about 
educational effectiveness, but in making that argument, they are accepting 
a standard of evidence far weaker than we routinely require in areas 
outside of education. The methods for evaluating educational programs are 
baracally the same as those used to evaluate, for example, medical 
treatments or health programs. In those fields, data similar to the NAEP 
would not be considen^d adequate for readiing conclusions about 
effectiveness or safety. If we accept weaker evident in the area of 
education* we may be less likely to do someone bodily harm, but we are just 
as likely to be wrong. Just as looser standards of evidence would sometimes 
cause medical practitioners to continue ineffective treatments or to 
terminate efifective ones, such standards would likely mean that some 
ineffective educational programs will gain credit and be emulated, while 
some effective ones will be Uireatened. 

The NAEP could be redesigned to support evaluation of educational 
programs, but it would require wholesale chaises in the program. 
Moreover, the changes that woulJ be called for--such as obtaining detailed 
information on family background, schod programs, and, especially, on 
students' achievement growth over time-are Hkely to be very expensive. 
Absent significant new resources, NAEP could not assume those additional 
roles without stibstantially weakening its collection of the t>asic achievement 
data that it needs fbr purposes of monitoring. 

ACCOUNTABIUTY 

The appropriateness of using the NAEP for aasmntability also arises 
in the debate about state comparisons and is a key consideration in the 
debate about the current statutory prohibition against the use of the NAEP 
for comparisons of local districts or schools. 

To some extent, using NAEP for accountability purposes would run 
afoul of the same difficulties just noted, for it is hard to bold •People 
meaningfully accountable for differences in educational achie ^ment if you 
cannot show that they are in fact responsible for them. There is, however, 
another reason to avoid using NAEP for accountability: it could undermine 
its value for monitoring. 

The key issue, as I stiggested earlier, is teaching to the test. Whether, 
and under what circumstances, teaching to the test is instnictionally 
desirable is the su, ject of an intense debate in the education community. 
For present purposes, however, you need not be distracted by that debate. 
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WTiatever the instructional danger-cr value- of teaching to the test, it can 
seriously undermine the validity of test scores as an index of achievement. 

Some proponents of using NAEP for accoimtability maintain that 
NAEP incorporates protections against this kind of corruption. It does. As 
currently operated, NAEP provides considerably more test security than do 
many other testing programs. In addition, it uses a sampling system in 
which each tested student is given only a fraction of the test. Both make it 
more difRcult for teachers and students to learn what the test contains, 
thus discouraging inappropriate teaching to the test It is by no means 
clear, however, that these protections are adequate. Even though NAEP 
tests are collected after use, many thousands of students and a good many 
teachers see the test while it is administered, and others see it for other 
reasons. Over time, knowledge of the content of the test will grow, and at 
that point the corruption caused by teaching to the test can begin. 

Local comparisons are far riskier than state comparisons in terms of 
potential corruption of the NAEP. In all but the smallest states, NAEP 
state comparisons leave most students and schools untested. Thus the 
people who are responsible for much teaching to the test^-Iocal educators- 
will oflon have no direct incentive to raise scores. State administrators will, 
but they have fewer means of pressuring local educators than in a system 
where the students of local educators are tested. Local use of the NAEP 
would dramatically increase the number of people whose perfoinnance (that 
is, whose students' performance) will be measured and will thus greatly 
increase both the incentive to raise scores and the means of exerting 
pressure to do so. 

Exactly so, respond the proponents of local use of NAEP: it would give 
people more incentive to improve their performance. It indeed could, but at 
the cost of threatening the fundamental validity of the descriptive 
information that NAEP was created to supply. 

This dilemma has a solution: use the NAEP for monitoring, and u.«^e 
other tests for accoimtability. If proponents of local comparisons-and the 
audiences who would get the results of them-were convinced of the 
necessity of separating the accountability and monitoring ftmctions of testa, 
it might be possible to have local reporting of the NAEP while not using it 
for accountability purposes. Under current conditions, however^ that 
prospect is unrealistic. Therefore, the prohibition against local use of NAPIP 
results is still needed to maintain the NAEP as an instrument of monitoring 
rather than accountability. 
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THE ROLE OF THE NATIONAL ASSE^MENT GOVERNING 
BOARD 

I would tike to make only a few comments about the National 
Assessment Governing Board (NAGB) in the context of the third of my 
general points: Uie unavoidable technical complexity of the NAEP. How 
should the NAEP be administered to insure that the technical ramifications 
of issues such as state comparisons and the use of performance assessments 
are acknowledged and handled appropriately? 

Under the current statute. NAGB has extremely broad authority over 
the design and conduct of the NAEP and over the reporting of NAEP 
results. This authority includes issue that can be resolved without a great 
deal of specialized expertise, such as selecting the subject areas to be 
assessed. Indeed, some people have axigued*-reasonabIy, in my view- that 
decisions of this sort should not be considered the province of experts; 
although experts sboiild be consulted, these issues are appropriate ones for 
wide public debate. On the other hand, the authority of NAGB also 
includes issues that are technically very com]:dex, such as "developing teat 
specifications ' and "designing the methodology or the assessment/ 

Under current law, NAGB is not constituted appropriately to deal with 
technical issues of this sort. Only two of the 23 seats are set aside for 
experts in testing and measurement. NAGB has also not used the leeway it 
does have under current law to augment its own hmited t^dinical expertise. 
For example, it has not constituted a standing panel of technical experts to 
advise it. (By contrast. ETS. as prime contractor for the NAEP, does have 
such a panel, despite the impressive expertise of many of its own staff.) 
NAC B does make use of outside technical consultants, but only on an ad 
hoc basis. These ad hoc arrangements do not provide a regular opportunity 
for experts to meet and deliberate about mGUor issues confronting the 
NAEP. 

The contrast between NAGB's level of technical expertise and that of 
other government-funded groups with authority over technical decisions -for 
example, NIH or AOAMHA review panels-is striking. We would not 
assemble a panel to design clinical drug trials that included only a few 
people with technical expertise in designing clinical trials, yet we have done 
precisely that in constituting NAGB. 

This lack of expertise is extremely important for several reasons. 
First, m educational measurement, as in other complex technical fields, 
there is no clear way to separate technical and substantive questions. In 
pharmacology, "merely technical" considerations determine whether the 
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medication you take will cure you or harm you; in measurement, 'merely 
technical*' considerations determine what the results of a test mean. As 
presently constituted^ NAGB does not have the ability to resolve »ome 
important technical questions. More important, in some instances, it may 
not have the expertise needed even to judge when technical issues are in 
fact imporUmt. To pick only one example, the oi^^isation charged with 
overseeing the NAEP should be able to recognize the need for validation of 
new assessment methods before they are fielded and reported on a national 
basis. 

Second. NAGB needs technical expertise in order to be able to respond 
to the increasingly extreme political pressures being exerted on assessments 
by various constituencies. These pressures are often very intense, and the 
constituencies involved appear to have ever less patience for technical issues 
that might slow changes they desire. Expertise in measurement is 
sometimes needed to understand which political pressures are reasonable 
and to find technically sound wa>^ of accommodating them. 

The quality and effectiveness of the NAEP therefore could be improved 
by changing the statute to insure a reasonable representation of technical 
expertise on the National Assessment Governing Board. Certainly, NAGB 
needs to incorporate perspectives of other sorts, because the goals of 
education are property a matter of public debate, not the sole province of 
experts. The design of assessments* however, does require specialized 
expertise. 

The extent to which the membership of NAGB shotild be modified 
depends on its purview. If NAGB retains authority over the highly 
technical issues it now oversees, its membership should be altered 
fundamentally. If» on the other hand, authority over some technical issues 
were removed from NAGB's purview, more modest changes might sufEce. 
Even then, however, NAGB would need sufficient technical expertise to be 
able to judge the importance of technical issues and to appreciate the 
technical ramifications of initiatives it cor *r8. 

Among the steps that might be taken to strengthen NAGB are these: 

o Increasing the number of seats (of the 23 total) assigned to 
technical experta in testing and mt*asurement to provide an 
eflFective voice for technical concerns. With NAGB's current 
purview, a reasonable level might be roughly a third of the 
Board-say, 8 of 23 seats. If NAGB s authority over 
technical decisions were reduced, it might suffice to allocate 
fewer seats lO technical experts- perhaps 4 or 5. 
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o Requiring that the individuals chosen for those seats be 
widely acknowledged as experts by their professional peers. 
There is currently no such requirement in the law. NIH 
and ADAMHA review panels might provide a useful 
example of how such expertise could be asstired. 

0 Requiring that NAGB constitute a standing advisory 
committee of ex|^rts in testing and measurement. This 
committee should meet several times a year and should be 
empowered to issue reports to NAGB, NCES, and other 
interested parties to the extent the committee deems 
necessary. This committee could presumably overlap with 
the technical advisory committee set up to advise the NAEP 
prime contractor. 

CONCLUSION 

The NAEP is currently at a turning point. With the currently 
authorized state comparisons, NAEP is taking m^or steps toward the 
functions of evaluation and accountability. Proposals for further changes in 
these directions have been on the table since the last Congress* and as the 
reauthorization of NAEP nears, pressures for further changes will intensify. 

My own view is that the changes made to date were undesirable and 
that further changes in those directions-sudi as the use of NAEP for local 
evaluation and comparisons-should be avoided. Absent dramatic ch'mges 
in the structure of the NAEP. the efforts to use it for evaluation are f\itile 
and potentially misleading, and efforts to use it for accountability threaten 
the NAEPs traditional role as the nation's only frequent and nationally 
representative indicator of the achievement of our youth. 

The Committee will hear-indeed, has already heard-fundamentally 
different views of what NAEP should be, and you may well decide that the 
traditional monitoring function is not in your view the most important role 
the NAEP can play. I would ask you to bear in mind, however, that despite 
some contrary claims, we cannot have our cake and eat it too, and pursuing 
evaluation and accountability will at best require short-changing the 
momloring function and at worst wilt undermine it. 

Finally, I would like to reiterate that providing the information on 
achievement that policymakers and the public wants is a technically 
complex and difBcult task. If the NAEP is to be successful as it evolves 
further, it must be given a governance structure that is cognizant of these 
difficulties and able to balance them against legitimate political pressures. 
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Mr, Chairman, thank you again for the opportunity to discuss these 
issues with the Subconunittee. At this time, I would be pleased to turn to 
any questions that you or other members of the Subcommittee have. 
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Chairman Kildee. Thank you, Dr. Koretz. Dr. Linn? 

Dr. Linn. Thank you, Mr. Chairman, for inviting me to testify 
today. I, too, will try to be brief and ask that my written comments 
be admitted into the record. 

Chairman Kildke. Without objection, they will be included. 

Dr. Linn. Many of the points that I want to make have already 
been made, and I would like to just reemphasize a few of those. 

To begin with, NAEP was designed, as has been pointed out sev- 
eral times, not as a test in the usual sense that you give to get 
scores on individual children or on individual schools but as an in- 
formation system to monitor prc^ess of the Nation's youth in edu- 
cational achievement over time. 

It has served us quite well over the last 20 years. We haven't 
always liked the messages thftt it has provided such as achieve- 
ment being—the trends being rather flat and at levels that are 
below those that we aspire to as a nation, but nonethele^ it is the 
best source of information that we have to have an independent 
monitoring of what is happening to achievement in the Nation. 

The current legislation has prohibitions against reporting scores 
for individual students or for individual schools. I would strongly 
urge that those prohibitions remain in effect. I think it is just im- 
possible for the system to serve both as a monitor of trends of 
achievement for the Nation and as an instrument that will report 
scores for individual students or for individual schools. 

Currently, the sampling of exercises for an individual student is 
quite slim, too slim to provide reliable scores for an individual stu- 
dent. To have reliable scores would greatly increase the amount of 
testing that would be required as well as distort the function of the 
monitor as an independent monitor. 

The same could be said for individual schools. By sampling and 
giving different exercises to different students, you can get a good 
reading for the Nation as a whole or for States as in the case that 
you now have permitted so I would strongly urge again that that 
prohibition against going down to the student or school level 
remain in effect. 

One of the mcgor change that took place in the 1988 legislation, 
of course, was the provision to allow the voluntary participation of 
States in the national assemment. In 1990 in accord with that for 
the first time, 37 States did (mrticipate in that in eighth grade 
mathematics, and in 1992 a large number of States plan to partici- 
pate in both reading and math at grade four and mathematics 
again at grade eight 

Congress in authorizing the trial State assessment put in place a 
requirement that there be an evaluation conducted. That is cur- 
rently under way under the National Academy of Education s guid- 
ance. I have the pleasure of co^hairing that panels and that panel 
will be reporting in October as miindated by Congre^. 

Because Congress needs to otnakler the autiiorization at this 
point in time, the panel will be submitting an interim report in 
about three weeks. I cannot at this point give you the details of 
that report because it is still under review, but f can say a couple 
<^ things. 
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First, we have not identified major flaws in the administration of 
the National A^essment at the State level that would preclude the 
relea% of the scores in June. 

Second, we have not seen adverse effects on the National Ass^s- 
ment itself, and so will be recommending that some kind of trial 
continue in 1994. Why continue a trial in 1994 as opposed to a full 
operational program? In part because many of the— while you will 
have tested in two grades in mathematics and one in reading, there 
are still many things that will be untried at the end of 1992, much 
less at this point, in 1990. 

In particular, there will have been no trial with twelfth grade 
students, and it is likely that just because you are successful at 
eighth grade or fourth grade, it does not necessarily follow that you 
would he successful at twelfth. The motivation to participate on the 
part of students may be quite different at twelfth grade than at 
other grad^. 

Moreover, the success on trials at a couple of grades in only two 
subjects in one, and one subject in another, is far removes from a 
full operational program that would involve three grades and three 
or four different subject areas, so we think that there are still a 
number of i^ues to be considered. 

In addition, there are parts of the trial State assessment that is 
limited to only public sch<K)I students at the current time, and if 
the goal is to really have comparisons of performance; of all stu- 
dents and States or States to the Nation, the number of students in 
private schools varies considerably from State to State, and that 
would be something that the Congress might want to consider. 

Finally, returning to national NAEP, I would like to point out 
one of the things that the original designers of National Assess- 
ment had in mind, and that was the assessment of all of the Na- 
tion s youth at a given age level. 

Early on, that included students in public schools, in private 
schools, and youth who had dropped out of school. Due to budget 
considerations and constant dollars, the out of school youth are no 
longer a part of the National Ass^ment. This is particularly both- 
ersome at age 17 where, in fact, we are a^essing only those stu- 
dents who are in schools and missing about a quarter of the 17 
year old youth that are no longer in school. 

So I would urge consideration of the need to know how not only 
students in school are doing but 17 year old youths in general. 

I was asked to comment on the National Asse^ment Governing 
Board. I claim no expertise in government so I will not make any 
recommendations there. 

I will make one observation, though, and that is that there seems 
to be some ambiguity in the degree of responsibility and authority 
between the National Assessment Governing Board and the Secre- 
tary of Education. I think that some clarification of that would be 
desirable. 

Thank you, Mr. Chairman. 

[The prepared statement of Robert L. Linn follows:] 
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Mr. Chairman, thank you for the opportunity to provide 
testimony to the subcommittee oversight hearings regarding the 
National Assessment of Educational Progress (NAEP) - The hearinqs 
and consideration of legislation regarding NAEP have particular 
Iniportance because of the current national attention that is 
being given to educational goals and the role of assessment both 
as a means of monitoring educational progress and as an 
instrument of changr. 

The inclusion of the word "progress" and the choice of tho 
word "assessment rather than "test" in naming NAEP were not 
aitcidental. When the original plans were being developed lor 
NAEP roughly a quarter of a century ago by people such as Ralph 
Tyler and the late Francis Keppel, NAEP was conceived of^ not as 
a test in the familiar sense* but as an infoxT»at ion system, one 
that would yield indicators of educational progress. The Grot;!. 
National Product and the consumer Price Index were used as 
analogies to the type of indicator that was needed to monitor 
progress in the educational achievement of the nation^s youth. 

The role of NAEP as an independent indicator of educational 
progress is quite different than other roles, such as school and 
teacher accountability or the certification of individual 
students, that tests are expected to serve. NAEP has provided 
the most reliable single source of information about the trends 
in achievement of the nation's youth over the more than twenty 
years since the administration of the first assessment. Although 
we may not like some of the messages that NAEP has provided (e.g. 
levels of achievement that have changed relatively little during 
the past twenty years and remain below those to which wo as a 
nation aspirt), we have no better way of monitoring educational 
progress for the na-ion as a whole. 

There are, of course, other roles for assessment. Thii; i :■ 
apparent in the cu^'itnt debate regarding the possibility of «i 
national examination system or a national test. it is, however, 
critical that the different purposes be clearly distinguished .md 
that the assessment systems be designed to be consistent with 
those purposes. 
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Prohibition Against Ranking, comparing, or Evaluating Individual 
StudeatB, aobools, or Schoo^ Districts 

NAEP was not designed to be a test that vould provide scores 
for individual students or schools. Indeed, such uses are 
precluded in the current law which requires confidentiality of 
individual student identification and of results for individual 
schools. Ranking, comparing, or evaluating, individual students, 
schools, or school districts is also prohibited. 

1 strongly urge that individual student and individual 
school results continue to bo kept confidential in futuro 
legislation. An expansion ot NAEP to provide results for 
individual schools or students would lead to a loss of NAtiP'is 
value as an independent and uncorrupted indicator of educat ioni'i 1 
progress. 

Since? each student tJikos only a sample ol the exorcisct: ur,ct.J 
at any given administration, NAF.P does not collect enough d.it»i to 
produce reliable scores for individual students. The 
administration of different samples of exerciseis to different 
saropXes ot students is a strength tor purposes ot obtainin^i a 
broad indicator ot ptTfotma nee for the nation or a state, but 
this use of multiple samples of exercises would be a d i ;;.jdvant 
for producing scores; for individual students - 

The r€*porting of iej;ults lor individual .sch.*olG would 
greatly incrcar.e the nise of the sample that wrii.ld be required at 
each participating school. More importantly, the focuis on r.f*ho<^l 
building result!; would increase the stakes that are attached to 
HAKP results, degrade the security of NAKP ex<'rcisc3, and 
increase the likelihood th<^t results would be corrupted. iJuch .i 
change would threaten the integrity of NAEP as an indi^pendent 
monitor of national or rotate trendir. in student achiov^^mont . 

Thi? extonsiuii ut NAKP beyond the state love) to th<* 
dii:*trict level rair,e:; a somewhat different r;et of is;;ues. 
Although 1 d^ not advocMte an expansion of NAi:r that would ailc^w 
voluntary p.j rt i t: ipat ion by largo school tiir.t rict r., I Ih»1 iovo X\\ \\ 
the case tor allowing participation may be as v.trong for larq'» 
.*;chool difstricti; .^s it is tor individuiil i;tatet>- Ihc ouf ifHt 
design which requires the assessment of only a sample comvisl \ 
of a relatively small traction ot the students within a grad** 
applies as readily to large districts such as IjOw Angelor., 
Houston, or New York City as it does for a number of r.tato:-.. 
Indeed, a r>unber ot distrit'tt; would havo more r.tudetits. than 
ot the less populous states. lk»cause of its special :;tatui;, < nu- 
dist rict» the District nf CoHmbia, did participate in t ho 
Trial state Assessment. Consequently, Jt seems just as 
reasonable and feasible, at least on a trial basis, to allow 
large districts to participate voluntarily in NAKP and roceiv<* 
district level reports as it is to provide this option to states;. 
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If the prohibition against the voluntary participation of 
school districts is reinoved, I believe it %*ould be wise to do so 
on a trial basis during which time the practice could be 
evaluated much in the way that this is now being done for the 
Trial State AssessEDent. I also believe that it would be wise to 
maintain a restriction for districts with enrollaents below some 
prc-spccif ied level. 

Trial State Assossaent 

One of the most significant changes in NAEP introduced in 
1988, when Congress enacted Public Law 100-297, was the 
introduction of the Trial State Assessment (TSA) program. 
Consistent with the provision of the law, NAEP made available to 
the states an eighth grade mathematics assessment in 1990. 
Thirty-seven states, the District of Columbia, and two 
territories vol?intarily participated in the 1990 TSA, Results ot 
that first TSA are scheduled for release in June of this year. 
Planning tor the second TSA, which will be expanded to include 
both reading and mathematics at grade four as well as mathem.^t icn 
at grade eight, is well underway. 

In accordance with the mandate of the law, the National 
Contc'r for Educational Statistics awarded a five-year grant to 
the National Academy of Education to conduct an independent 
evaluation of the Trial state Assessment. That evaluation is 
being conductod by an eighteen member panel with a wide range ot 
technical and policy expertise that was appointed by the National 
Academy of Education. I have the pleasure of serving as co-chair 
of the panel with Profcsnor Robert Glascr from the University of 
Pittsburgh. 

The panf»l is scheduled to report on the evaluation of the 
Triai State Assessment in October, 1991. A second report, 
which will include an evaluation o*' 1 *th the 19^0 and 1992 Trial 
tltate Asse:5smpnts, is scheduled for late fall of 1993. The panel 
has commissioned a number of analyses and papers that will, as 
the law requires, "assess the feasibility and validity of [state] 
assessments and the fairness and accuracy of the d»ita they 
produce • " 

Although a complete evaluation of the Trial .State Assessment 
at this t ime--alinost three months before the first state rer.ults 
are scheduled tor release — is not possible, it is recognized that 
Congress needs to consider authorizations now if state 
assessments are to be continued in 1994. Hence, the National 
Academy of Education Pcinel will submit an interim report within 
the next few weeks. 

Because the interim report is still under review, I cannot 
provide details of that report at this time. However, I can say 
that the panel has not identified major flaws in the sampling or 
administration of the Trial State Assessment that should preclude 
the release of the state results as scheduled in June. 1 can 
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also report that the initial exporience was sufficiently posit iv** 
and without adverse effects on the national assessment that the? 
panel will recommend continuation of an expanded trial in 1994. 

Even at the end of the 1992 assessioGnt there will bo several 
aspects of NAFP that have riot been evaluated for use at the state 
level. Only fourth and eighth grade students will have been 
included in the TSA with assessments in only one subject area at 
grade eight and two at grade four. Only public school students 
will have been assessed in the TSA. 

There arc significant differences between high school 
neniors and students in grades four or eight in their likely 
willingness to participate in an assessment, their degree ot 
cooperation during the assessment, and thoir motivation. Thftw- 
ditfercnccs make it unwise to assume th.it a successful trial at 
e«iriicr grades implies success for a twelfth grade r.tatc-by-;.t .it i- 
assessment. similarly, successful implementation of state 
aj^scssments in one or two subjects; at two grades docs not 
4-»ecessar iiy imply that a tuli-scate, operational statt^-by-i:.! at e 
a£,r.essment in throe grades and three or lour subjects, cv>uld be 
conducted iiUccessfaUy without degrading the quality of the 
national assessment results. Finally, success with only publ it- 
school students does not guarantee success if both private ami 
public school students are included in state level asr,f?s.5mont i- . 
Hence?, it seems prudent to (*uthorij:e a continuation of an 
expanded trial through 1^>94 rather than a tuli-scaio 
implementat ion. 

Kocause there is? com- idorablo variability I torn st.ite to 
rotate in the proportion ot students enrolled in private schtio!:;, 
exclusion ol private school students t rom the state samples 
degrades the comparability of state- to-state rer-ults, A bott<»i 
indication of the achievement of the student t; within a statv 
would be obtained it both private and public :.rhociI lUudcnti: w.-rr 
included in thr state administrations of NATl . Thcrotore, 1 
believe that provisions for the addititm ol pr v.iti* luhool:. to 
the state samples deserves; consideration. 

Out of School Youth 

In the early year;: ot NAi:r, a.';«e!;nmont s wor<^ admini r.t erf^ci 1 1> 
age cohort samples, both in and out of school. Hue to drcltnini 
riupport in constant dollars, the inclusion ol out-ol -!;<.*hool 
13, and r;-y<*ar-oidr. was lost. This lor,r^ ir, par t i cu 1 .* r 1 y noj ioa;-. 
for the 17-year-old cohort. 

It is important to know what hiqh r.chool t.tMiirui; iio. It 

ir, .tlr.o important to monitor the achievomont ot 17-year-oid 
jttudents who remain in school regard ler.;; of grade level . Kvf*n 
the latter, however, excludes roughly ^t>% of the nation's l7-yeai 
-olds who have dropped out of school . 
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The critical ity of including out -of -school youth in the 
assessment was stressed in the original design of NAEP. The 
rationale was clearly articulated by Ralph Tyler in 1966. '*ln 
our total society, there are children in public schools, in 
private schools, and some who are not in school. Hence, a 
comprehensive assessment pxrogram requires samples which include 
children who are found in these several places* ** 

I believe that Tyler's reasons for wanting out-of -school 
youth included in the assessments are, if anything, toore 
compelling today than they were a quaiter of a century ago. The 
use of college admissions tests as indicators of educational 
progress is much maligned, and rightly so, in part, because these 
tests are taken by only a selectcsd and changing segment of the 
population that is planning to go to college. Although losr. 
severe, NAEP is subject to a similar criticism, especially for 
the IV -year-old cohort. 

The reintroduct ion of the assessment of out -of -school youth 
as a requl.ir feature of NAEP would provide tor a substantial 
improvement of the quality of information that NAEP provideis 
about the educational progress of the nation. For purpoccy of 
providing a n.itional indicator of achievement, a kind of Grorr, 
Educational Product, I believe that the inclusion ot out-ol- 
achooi youth deserves high priority. 

National Assessment Governing Board 

One of the topics mentioned in Congresnman Kildee'r. li»tter 
ot invitation to testify was the role of the National AGCOi^r.menl 
Governing Board (NAGB) . Since i claim no special expertine on 
the subject of governance or bureaucratic organi i;at ion. 1 will 
not make specific recommendations regarding the role of HAGB. l 
will make one observation, however. 

There appears to be considerable ambiguity in the oxlont ot 
f^AGU's responsibilities and authority vs. those of the Secretary 
of Education. Clarification by Congress seems desirable, and 
once clarified, it will be important that funding is commc-njuratt- 
with the der>ignated responsibility and authority. 

Hesprct fully £?ubmitted by Robert L. Linn for the March Xi, 
109 1 hearing of the Subcommittee on Elemrntary, Secondary, and 
Vocational Efiutation of the Committee on Education and U^bur, U. 
r». House of Representatives. 
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Chairman Kildee. Thank you very much. 
Dr. Roeber? 

Dr! Roeber. Thank you, Mr. Chairman. I, too, request that my 
written statement be entered into the record. 

Chairman Kildee, Without objection, it will be entered m the 
record. 

Dr. Roeber, I also apologize in advance if my voice fad^. I came 
down with, I think, a case of the flu coming back from the 
Coast at a meeting last week, and sit here in a rather weakened 
condition. I wish I was stronger so I could speak out more forcibly 

on this topic. ^orrn^ 

As a director of a^essment in Michigan sin(» the mid 1970s, 1 
have had the opportunity of working with States in their pn^frams 
and helping to shape State programs since the mid 1910b, but since 
1 began my career working for National Assessment in the early 
19708, I have seen National Assessment from a personal perspec- 
tive and can bring that to your discussions. 

I support the current shape of National Ass^ment as a monitor 
of national achievement in our country, and I feel strongly that 
that is an appropriate role to continue for National Assessment. 

I, too, have concerns about the ways in which a State NAEP, a 
district NAEP. a school NAEP. or a student NAEP. might unfold. 
These deal with the substance of National Ai^essment as well as 
the process for arriving at that. 

First, the process concerns. 1 am concerned that national pro- 
grams such as the National Assessment or the othere that you will 
consider tomorrow are being developed without the involvement of 
State level curriculum and assessment persons. We have had an as- 
sessment program in Michigan for 22 years, M<^ of the lessons 
that are now being learned and/or rediscovered at the national 
level we have learned the hard way in Michigan, and that is one of 
the reasons in the mid 19708 we formed the Association of State 
Assessment Programs so that we could share thc^ hard knocks 
and prevent othere from going through that. 

I think the notable exception to this has been the consensus proc- 
ess that the Council of State School Officers has been using to 
arrive at the assessment frameworks. 

Most specifically, we have objected to the ways in which the iMa- 
tional Assessment Governing Board has adopted plans for the 
future which call for sweeping changes in the scope of National As- 
sessment without State involvement, and certainly the technical 
expertise that was referred to earlier is something that is d^per- 
ately missing from that so that I have felt strongly for a couple of 
years that practitioners such as State curriculum people. State test- 
ing people, as well as our counterparts at the local level, need to be 
heavily involved in the design of the National Assessment or other 
National Assessment pn^ams. 

We have teen involved to some extent in the administration of 
the State National Assessment program, but we seem to be most 
involved in the trivial aspects of the administration and almost 
completely uninvolv^ in the design, in the shaping of the policies 
that underlie it, as well as the procedures for assessment, i think 
we have much to offer. 
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Now my substantive comments about the future of National As- 
sessment. First of all, I am concerned about the quality of the as- 
s^ment materials that are used. There is a trend now nationally 
to incorporate performance assessments in assessment, and I ui^e 
the National Assessment Project to go back to the future. 

When I worked in National Ass^sment in the early 1910s we 
had individual performance testing in areas as diverse as art» 
music, science, social studio, career and occupational development, 
mathematics, and writing. We literally asse^d 40,000 or 50,000 
youngsters in singing, playing musical instnmients, and ended up 
with about 80,000 tai^ which we scored, and so the National As- 
sessment Governing Board s comments that relate to the use of 
performance assessment, that it is untried, and that we naed to go 
slow on this, to me are simply not a recognition of the heritage of 
the project that they are si'pervising. 

Second, 1 think the m<^t valuable aspect of National Assessment, 
it is an original design. I think a real testimony to Ralph Tyler's 
inspiration in designing this program in the mid 1960s was the in- 
clusion of an ass^ment of young adults, age 26 to 35. 

Almost no one knows about that l^^ause it was dropped in the 
early IDTOs because of Federal budget concerns. The reason why I 
feel that this is the most valuable part of National Assessment is 
because what ultimately we are concerned about in this country, 
and rightfully so, is whether our high school graduates, our college 
graduates, and even thrae with advanced training, are prepared to 
enter the workplace and \^ productive citizens. 

What better way to determine that than, in fact, to carry out an 
assessment of these individuals? 

I think that, once again, it demonstrated the inspiration of the 
early design of National Assessment that this 20,000 to 25,000 stu- 
dent or adults in that age range couid be tested on an annual basis 
in an appropriation, incidentally, that is about comparable to what 
it is now, unadjusted for inflation; that is, the appropriation in the 
early 1970s was about $6 million to $7 million a year, and they 
were able to do this individual performance testing that I referred 
to as well as that of the young adults. 

Third. I am concerned about the content areas that are assessed. 
There seems to be an over-emphasis in my mind on mathematics, 
reading, writing, social studies, and citizenship, and I would like to 
see the National Assessment branch out into some other areas that 
I think are vitally needed, including the arts, foreign language, and 
health and physical education. I am sure that there are other 
areas. 

Fourth, as I look at assessment above the level of the State, I am 
most concerned about whether and how Michigan performs rela- 
tive to the world. You know, the automobile industry is not con- 
cerned whether the autos built in Indiana or Ohio are tetter than 
the ones built in Michigan. We are concerned about building better 
autos than the rest of the world. The same ought to be true with 
student achievement. 

So rather than driving NAEP downwards into our schools and 
districts and even the State, I am concerned about driving NAEP 
upwards and linking to international assessments. We had the op- 
portunity. Unfortunately, we couldn^t come up with the funding. It 
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would have cost the State $120,000 to link the Michigan reading 
trat at fourth grade to the International Education Achievement 
reading literacy test at fourth grade, and I think that would have 
provided data for every school and every district on the lEA Read- 
ing Literacy Scale for $120,000 without having had to give that test 
to all of our students. 

There are ways that we ought to be exploring to link the ass^s- 
ments without using one to drive out another, to link them togeth- 
er so that one a^essment is essentially carrying the load of two or 
three, and, in fact, given what I could have gotten out of the lEA 
Iteading Literacy Study, that is school, district. State, national, and 
international data, I really wondered whether we needed the Na- 
tional Assessment State by State data at grade four. 

School and district testing on NAEP? It is certainly a controver- 
sial issue, and there are Stat^ that are in favor of it. There are 
States that use commercial available standardized achievement 
t^ts, and they would prefer to use the National Assessment. Other 
States, such as Michigan, find less value in that, and I understand 
that that is an issue that I think you are going to be wrestling 
with. 

But I think that I personally am in favor of lifting that prohibi- 
tion so long as all of the principal parties at the otate and local 
level are in agreement to it so that it is not forced on someone. It is 
left on a voluntary basis at the State and the local level so that if 
the local district cares to get that information, they can, and if 
they choose not to, they also can avoid that. 

Frequency of testing is an i^ue that you are going to be also dis- 
cussing. I agree that annual assessments would be desirable, but I 
disagree about a two year cycle. NAEP used to be on a four year 
cycle, and I think that is frequent enough to see the kind of 
changes that we expect on a global basis in achievement in stu- 
dents. 

Programs such as National A^essmeiit have been quite able over 
cyclra as long as five or six years to track trends in student 
achievement because, again, as a monitoring program testing every 
four years or five years is appropriate. 

If the purpose of NAEP is an accountability prc«ram to force 
schools to change, then clearly four years is way too long, but I be- 
lieve that NAEP's greatest value is a monitor. 

The narticular issue of the motivation of twelfth graders has 
been alluded to, and it is one that urges me to urge you to continue 
the trial nature of National Assessment State assessment programs 
for 1994, 

Twelfth grade was selected as the third of the three grad^ to be 
ass^sed. Testing takes place in February. Anyone familiar with 
schools and how they operate will indicate to you that seniors in 
February are not very motivated. 

Now, 1 am not particularly concerned about whether Michigan 
students are more or 1^ motivated than Indiana students so in a 
State comparative program, I don't care. But when people are 
making absolute judgments about what our high school graduates 
have at the time they leave school, then it is a critical issue. 

When National AMessment was assessing 17 year olds, thc^ as- 
sessments, the modal grade level at which those assessments oc- 
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curred was eleventh grade. Now, that is not quite as neat as having 
a picture of students as they leave school, but right now I am very 
fearful that the twelfth grade data that we get is seriously underes- 
timating what our students know because of the lack of motivation 
of students to take tha test. 

The final issue I want to talk about is whether testing can 
change student learning. Obviously, as a supervisor of a program 
that has been in business for 22 years, I believe it can, but will the 
test itself cause schools to change? We have a long history in 
Michigan of knowing that it won*t; that in fact you need to engage 
classroom teachers in the prcK^ess of change through local school 
improvement programs, through the provision of instructional as- 
sistants, model instructional programs, and even mwiel curriculum. 

It is a lengthy proce^, one that involves considerable technical 
a^istance to be provided to each and every school that is partici- 
pating. 

What has concerned me about the 1990 and the 1992 trial pro- 
grams is that we are proceeding as kind of a mixed mwiel. NAEP 
is a monitor, but NAEP is also for school accountability, and when 
we get to the actual **what are we going to do with the results,'' 
there seems to be a hands-ofT approach at the national level, that it 
is not appropriate to develop suggested instructional strategies in 
mathematics or even to indicate to the public that a revamping of 
mathematics instruction is nec^sary. 

Now, somehow it is okay to test students and beat States and 
schools up, but it is not okay to help them, and. in fact, there has 
been no funding provided for the assistance that States and schools 
are going to need to improve their instruction. I and others in the 
assessment and curriculum groups have taken it upon ourselves on 
a voluntary basis, sometimes at other agencies* expenses, to try 
and develop resources that States can use to interpret the National 
Assessment results. 

In separate correspondence to the National Center for Education 
Statistics, I have recommended that at least five percent of every 
assessment be set aside for the kinds of technical assistance and 
improvement activity that I believe should occur after every assess- 
ment. 

Mr. Chairman, I appreciate the opportunity of addressing these 
concerns to the committee, and I like the others would be happy to 
answer questions. Thanks. 

{The prepared statement of Edward D. Roeber foP-^ vs:] 
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Mf Cnctm^cn mofntKns of the commmoo. diSltngtjish»<3 poneists ana gviests i oiii 
ttlward D J?oet>ef fluporvisoc of tr^ Ass^sim^ tynJ Accr^to?>oT'> Pfogroms Tcm th»? 
MJcNQon neportnrjent of Educaiw I otio Co Ct>of of tn© Assortcrtton of State 
Ass<»sment Procjrom& o tecfvrtca} ossstonce network omong ^he indvjduott wUo wt>'W 
tn Stat© ogenoes f»sixv\sibio fo« conductrvg voftous stote levoi stuci^nf ossessm^nf 
octMties The f>etwort( ^kh served OS o commuracotton too* among sTot<?s to o»*1 o^*"^ 
onothor with the deveiopmenf ond tmpiom&nlohon of owx stoif© ossesirr>en! p<ogfoms 
me r«twori» ^kos e5tQt3*fehe<t «n *976 becous© slates were consicfefirtg Qfxi 
rr»f>w>fnerrt»r^ o vO»l€»tv Of $tote »©v©i p<ogfom5 cfxJ w© tett thai try wofkfog toget^^e^ we 
coiAi help one orKJthef *n *rr^>ieme«t'r>Q otif progfoms m the tT^ft rr>onf>ef c>05S4t>ie Irve 
need fOf th!$ technicof ossiJtonco netwofk has ooJy »ncfeC5s*KJ fn fecef>i v^KM. g-ven the 
t:Jerr»<yv3S tc r>ew r^nQroms ct the St<7te Jev*»> the r oto fc* rpfofm «n tt>e notiJfO oJ 
ossessnwit as weB o$ the new assessment pioofoms dnveiopftd ot txnno ci»k usso<i 
notonoltv 

At the outset s wtV^ to exD^oss to the cornm*ttee the group s a;:^ec. kOUin fa 
oppoftunffy { have boeo given to spOOfc c«X>ut tt\e Nottonoi Assessment o' f O'jcotfOfXJ' 
Progiesfi (NAtP) OS tt>e a«co$siOo the continuatfon and ex^KJ/wrt of NAfP Joi.es r>''H 
} wor>t to ftfst expiess the Supporf o( the grot^/p myseff nctucted fo( the tvp^ o' 
assessment which fs representee by NAf P from a peoortof pftfspectJve tfus supsxx^ 
QTOws out of tvjvirx} storted my cofeer m ossessmenf with three miri of wort* on tf »e 
NkJtJOhcS Assessment p^?ject when tt was t^Kle^ the f1»eclti>n the E<5uc at»on 
Commfss*oo of tt>e Stores OS wefl os my fifteen years work *n Mc^Tigon in oc^cJ^^or. » 
om sehrtng 05 o nr>embef o? the f i^otuat»on P<»>ef whtch Con^ss estoWiihod os po?* 
tt^e outt%oniat»on ot Notwxjt Assessment Trioi State Assessmertt pfOQ^ams to he 
con<tucte<? n >990 onrt ^ W {n oB of this eiepenefxre i t>eikeve tf>at tt^ere is a ct>nsiv I'v*-' 
roJe whjc^k o not fOhoi o&sessmont of stucJent feomtng con pk>v ^ improve*^e<-<t ^t 
Jeo^runQ oppOftun}t«»s toe Amef<on chrtdren and youth 1 bolieve that NAf P hos o^tro^iv 
ployed soch o fO*e ^n tr^ post ond can and ihouJd cofita^ue to do so m the ft/uie 

HowOwet f hove co^'e^ns w^n regafd to tt>e manner n whic:h the Nato^xr^ Assessm*v,r 
piOfect «5 C*e*ng planned ond <mpJementea as wef^ os. reg-cvdiftQ seve»a' bp**c»^»c ^sSi/ev 
Attorhed to th»s test*mnrLy ts o recent rrYemorondum that I wfote detoAnp trve conce"»< 
thct { hove oboi/t the pfocen of tmpiemenJtng the i990 ond ^ W tvjotfoooj As&essfT^ef^T 
ffiof State Assessment Pfogtoms os wes os the fjfonninQ foi t9<M orvd beyond As cof • 
be seen. \ cm mo?t concerned that the stote iewe^ os&essment persons tr»e ooe^ w 
wealth of edocatfono* techrkico' and pfrx'ttnii expenefKes OfO nnt reol'v r>vrwv*\5 
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ff^ hoipi»XJ ro from© nv? issues otxxj^ tf>© tuf^e of NA£P oeveioring opi>oo& fey «>och 
issue, nor decx^ng the di»SMot>ie opfcwi me$e corKre^m f©*qto onmoftfy to iho prc»ceu 
of invoCvvmvnl fixjt ^. that ff^ porsons wim the rworttcoi ©*pener>ce ot© not utt;t;^ 
!r» cSGSignof NAEP 

Ais<»ssmonf progfjm Iheso rcw>cefns con!©? on s<?v^oi matof oi^as. 

assessrnenf pfop/omj hos ©m«*»p«K3 os an o/{>a <^ reot nee<i n th^ fepst couD^e of Y<*n'i 
Ttvs r>a5 c<n«©cf cJeve^op^f? of Such pfofifomi, tno Nafioncn Assessm&nt G<>ve^r>.r»Q 
BoQfd (NAGB) tnckjcH^. to stody fx)^* juch os&essmonts ccn bo ^^i\x$e^ m ii^w 
pjo^/oms In tfx? Jong fonge plan tor NATP whrch NiAGB cxJoptecJ in Decombef , i<?89 cpht 
f€K3tfcrnfKJ n Nnverr»t<>r 1990, p<HtonTH3nc<> cnsessmenf ts montfOft^ w<Th o COUt^crKJfv 
note utQing fuHhof sHxJy orvl ccrc^fgi impJofnenfafion tfwxjgn a sertes of iPtiK'OS stuci.ps 

Wh4e ) m no? ofifx^ci to cofpt.* «Ttp4ompntaiK>ri of new as&pssment stfot^^os. wos 
NA[P that pK>r>eeTt>0 ^irn ow-.v-nents m the iok> JPftOs cv>d eoffy i9?o$ Wh.fe rM;>rH» o' 
Th;5 oo^y w4^k r€»rr»o^ tn NAE T m^fe Qt of>e ttme mdivfdk^oi ano q-Okip 
CH>ftcymartce cwismcmfs ofooi Svx:^ ax reodinp scence $o<:«» 5tuciJ<>s cv? muv: 
ccv&pr orxJ occopotKyvjf fJ#'vetf>pmenT oTKl so fcwtn Te»n of tfxx/sonrts of stu* v»f fs orv) 
young ocMts wf»'e tostpcJ wtth tno^e ff of^v pfog^am cc»utd r^^tf<J3y ck5.rrt tnn* %^cr. 
a»es5nH»nTs wt>»p f<K>vWe ona t<9f•^^^Mcoi^y $(Xin<l ts NAtP Therefore if iim^ to 

hCKesso^v t>e so coui^xis' 

A«©f$m©nt» oi VounQ Adutti )f a KvnAwnot -fcin*, , rngt fede^o* fu^ 'd& Ofo t)or>c5 *« ^ 
Of^v^iOp r»ew Stucf'^ Of Otliin frtoracy vwhen OfV* f^V* most vi3}vy(:«>io ov><*'"H o* NA( 
ff>*» cns<>$srn<»nf of ytx^ng CJCJu^ wos (JfOppoO m tf^ ^P/O^ *iv*€' to feciOfaf t)u'^igr>« 
CuH ft *j t,m<a fQ ro-nst.f uto thtf OMrf^Sir^onf oncJ ft *J.it*t tly tc> the ,n V'hcK)* 
assossm^fits of ^otorxjJ Asses^m<.nt NA^P hos oifOOOv p^Qvon the feosiCj-ifty or.si 
voJuo of CJSSo$j»r>y the f<n<c^uafs of srfyx>*in3 among yo^ifvQ odx^its Sue h data woov? t «^ 
jftwo^uOtvie ri rtfit<»rmjnif^> t?,e ft>o^*ne5i of Students fo^ <^Oi^gf^ O' si ro'{>*»-v V<:»»\<m - 
fo.'X3 «?pafOt<* StvKi.eS of oduf*S thfOugn tri^ Dep<.>rlnr>v^nT ,->f i ^ vv>rT>^ f-,/•,■»^ 

ar)pfor>«Ote fo coiry thew^ out fn a to$hton r fv^vKt^fHl to NAJ P 

Content At«m AssM&«<} Jhf* ^^ tir^ oth<>f fmj i^.rTo/'.t i-rpns s^<->uf' 3 tv- Aj'^^n 

trie f;al«or>oJ As^vinv>nt o# f.^t^^^p T^^j^ irvju.f«* ''•^ o^s ^op.pn Jong^cit?*^!, 
he^^Ith and phyVCOt f.TnPSs. to(,h of fhPSf* orpos hoi Of^n osiowed ^ or^e ot »rt,vp 
S'otf's ond 5ev(»fa{ «»oti>j ho>e Q$&eiSOd them u^-r\Q mdrvKJuoi o^d pHH/f> i^i't^mo- - 
^K^f'-ises in odd.t<in NAf p ^x>^ corKJvjrt^*^ two na'Ofvx fn^5V»<^r.t5 of tx-**- v -.i-j /j'* 
iiTH^ rruAic mckiO^ ffi^ p<*ffo*mcrx:e osm:ts of i><x h pfOQfCjm 

MoJhrofton o# l?*tv Gfoctert Cne tsK^e wtuch f>a^ t^e©n fois<*d fopeoiedSy tf»f 
motfvat>on of fvoWh gro^ft. Uf, kas P asses*mc*nt wt^en tr n gven *n f^to i^f 
^hftw senior ^h o COmpofOtf^^ pfogfom, ths i&s^je (s i^iS »mp<WoFit Aftoj otf tr><> 

sef tiors tn one stote pfOtXJWy won f tX* ony more motfvoted fhon those onothc-f 
Mtjwovm when tt>e oo^o 5 used jo q mof« oPsotyf e sens** io Joy who* pr opofiio^^ of 
seniors nppeof to Pe wf»9 f>fepw#*^i fo# roJfege of wcw ftv> jcicfc of mofrvofto-. uf . frv<> 
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osM«&mo->T con c<Xi$e undot es1»matO$ o' t^«w i€Kyrwr»g otid couW couie 

fxition to moke s^^ous <»"n« in foiiQ0n>en^ cJL>oui 1h© chcsn^jw whch must ma^e 
ou* eOucat*ono< Jvstems tn ofctef to ne^ $tiKJorti M?om whot tr^e.^ ne&a ^of futr^or 

ma Issue tJe5«»fv<»s spoc^oi sfiwy m«? studv ts n^9<S t>ofom fufincM ossf?ssn^ont w<vfc 
isctyrtmJCxitaigradotverve fn tf^e post wfi€?n stucJents osseisecJ erf 
4ev©f>feer> most ot tnose students orode eioven Corefvi study Ot student 

motvotton ts no^dod beforo we invest k) mucn m co8<K'tir>g gicxte ^etv© doto tnot tno 
»ongrtudH>3< trerxfe bocome so frrwlont that we cent chanye fegafd>P« of Mw 
fouiTv tr>e doto mov bo This is a seffous c^ooenoe to the inleortty erf th© doto wh<:h 
nwHis coT<»fui stixJy ^ ft^ ? 993 o» T9<M NAEP i ^^cve volunteered Mic^^€^on os o test sfe 
for tnis Study txjt fiov^nt been at3*e toconvlrxre the Nottooctf Centei for Educafortni 
^of sncs <^^CtSl of the need foi the stkxJv 

Aift«m*n»<rffheDWrtc1«KlSk:ncK>IUv#Jl Qnefecsture t>uift tr>to the lWOfv3 1W 
NotK>nol Asso«menf IrtcS State Assessmef^? tyogroms s tt^ pfohtD«K>n on ttv© lolfKjM- 
Of th© NAf P fosurt^ t>Y dfeifict . br txjadmg o» by siudent NAGB has pfcw^ed fff*'^ 
ty->n severed stcstes ore concerned oboi/t the import of NAf P wrtNn then stores onrf 
hove asked tt^ot th© txm not toe Wed Othof stores , wishing to »eptoce theff sto*ew»0e 
no*m refemncod testing prOQiam hove K/PPOfted Jffttng tfw bon Corr>p{K:otir>o ftvs 
issue ore the mode t>y the fotmet chc^f of NAGB t^^se remofvs cJeofty »nd»cate 

tne mtent o? NAGB to see thot on eve^y pop*J NAEP repioce the vat»etv of asscsirnont 
pfOQfoms whK:h e»is!s of the state and *cx:^ feveis 

It ts my feeitf^ f rwi the txtn shoutd t»e Ifted , but that ConQtess shooW do so *n o mor ^x-* 
wMtchsetsupopcocessbywhichiNsJsfOfTiodoaf otthestote^evei nus shouw be on 
vofLrntory octMty o? both the SiOte and focol levels to be cc^ned OUl of sfote Icx:o» 
cost and with the oppf ovo^ poftcymoke»s o* trie leve^ i*^e the octvrty ts to be 
cofTied oiJf Thfc meons the pCM^rnof . state bomS of educotion ona the Jeo*iofufH * 
the state feve* ond the boo^d educotton ond supeftfvterxJen? tocoBy 

m odd«kx^. ft>e NCf S. NAG© and the NA5P contiodcas shouSd be ojthoftred to conduct 
itotis!»c<i itf>hing sf ud^ where stotes ond kxat districts wtsh to enpooe fh such wort* 
This con serve the usetu* purpose of pfOVK*no on oppOfton*ty Id o State to feport 
Notionol Assessment scOfes wptho^ ho^ to Qfvo NMP tests to oB students, no» havtn^? 
to imk such tests onnuo»y wne^e poss*>te the test^wt»ch totes pioce of one fevo» 
shoiidnotditveo*ittestr>gatanott^er*evei rnsteod. the effc*eoc>es <^ bnking such 
pfog/ams needs to bo eirptwed ft ts $imp»y tr>c¥)piopi*ate to assunc^e ttyjt just because 
otftst IS odminstefed notionoffy that rt sbettet ffion those ©tven of tne stote of torot 
levetsf 

Fftt<9>tncv 0# T«S«oo NAGB hos proposed sh'ffinQ NAI P bock to on onnuoi cxssossment 

Pfogfom. asse»5»ng sb« subject oreos fotoi of three ofeos per 

As lonQ as NA£P IS retuhrig to ttw post « this retjords J strongv so{^^ 

cw;s*de^ onnoo? testr»g but on o ttvee or four-veof cvc*e Tha woukJ obow o (ongef 

development cycie (three Cf toof yeors msleod of tt>e coffeni two yeor^ more subject 

ofeas to be ossewed (we o* mae insteod of sa) and af»ow mo^e time fo* trends to 
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occur it cKni^ftu^ trwjt jjgfvticonf cNr^gei t\ sTudont p^ormar\co wfR occur %n jus? fwo 
yeo^s in tne ecFty veof5 of NAtP, ttvo<» or^dtour ylKJf cycles wyiy use<i to aftow 
cfKmgos *n $fud€wif pcwtofmonc© to occu* be^or© t€^ o&«»5Sm©nt TtT© ossejsmeof 
dftvotopmpnt cycle, cuirerrfty ct two ytw. ftsf is ix^t odaquotf* tof tr>© typr^s of 
cJevPtopmont. ri»Jof -tesfing 08Se9srnent. sco^tn^ Onci StOJVJOra setting w^cft tS r^fi^aort 
to conduct and roixvt on o quotfty 055<»$smDnt Atr%r«>^» Oft^Xff vf>af CyCie woiiUJ o'h>w 
mors time for such wortt to totf© p*oc© 

Unki to tRl«fl>odiona} Asymnwnft T>>e most exc^tng po^^tb^y wtvtch coutd r>e 
cc>ns)dOf©c1 ii cOTtcoptuo! Of sfofisticctf finks fo tnfenyjf ionot progroms of ossGSsmont 
Given the choice w?ip*hef to 'd^vcr* MAf P down vizards r>to states arKS foccif dtstrtcts o^ 
upwar«1& Jnto {ntenxjltorof oMossments the cfxwce ot^w to be ciec^ Gtve^^ tt>e 
Qtteotion bcNng focused on Intefnotionc^ economic cornpoWfven<H^ c d roJe of 
educcitton tn pfepo^trxi students fof futfior education ond tor viKXt. ft seems to me that 
ouf pftOfJty ougtit to be to detefTr*>e tKrw wefl our students ore dc^ng in compo^son to 
the students jn ofhe^ notons However, ttie current lntemolK>fKjt E durottonol 
Achievement (If A> studies Ore not connected »n onv woy w>th !^^e NiAFP stufitc^ ofhor 
ttvjn thot students tn focc< dfStncts ere tested twice 

Pother thon spend fe50ufC*?5 ond ef^e^gy tfymg to drrv^j ou! state and ?<x:tti tnstinQ 
piogtoms oi-Ki »*» p?oc© trvem with tStAf p (wh»ch WGB seen-^b be»il on d{w>g) S€»ems to 
me tfxjt ft wouW t>e muc*^ mofe truf^xi to enpfore fc^mof ttnfes with osse^*ment orogfvjms 
&)ternattonafV My goOJ wouk3L>e toet^^^XJte redundont doto coBet ttoo here too t>y 
lAsn^ comrnon dcjto CO«ecrKV> fochnta^^ 0( t?eftef yet tten<3tno pfog*ams togethoT 
As Jt sfc^KJs now o;x>roting two io* three) nottonoJ ond lotefncftionai ossessmof it 
P»ogfom5 1$ npt onty wostofuf ot federo? and state dotkjrs ff fs o^so wostofii? of t(>oc r^r 
onti studnnt ttm«. strice rr>ese redundont progfoms eoct . .TH^t be odnwxisttKoa kx <». y 
(ff> ofidition to stotf? and k>coJ ossevs^wtt pfogfon^s) 

U^fY a NAIP for Stale and Loco) C om pcglio m aj o p^^o^ who $t^<H>gfv surr'< -^s 
CTiterion referencml jntRrpretotons o* dOtO CSO(:f> OS 'Hove Oi hove no? J.t^HJe*»ts 
teomed wtx>t we wont tnem to leom'?') » om nor oif that fnrcfestcvl t> krx^wtng wt^^trxv 
Mchjgon scores htghe? or tower ttxjn Uvjiono c Oht?> Moweve*. NAIP con serve t->U\ 
purposes pOfttcuJorty {f th<> doto oie reported rf> terms ot percenroge o* stutK^f.ts 
ochtoving ot sotisfortory $eveis onrj th©n rpmponng those r^gufes OT'oss stti^ps ♦ 
P*>ff>ve thot tht^ fCHHTfltfig f 4arw»fi for »h<» iWf> Ino) t>(X}^'>m wt» rfn f,?t rhr^ ^, > t^>,ir i f,-.,/ 
that compo^otJve data wHj serve a uspfui putpose 

Th**ce wt8 t>e those who fnef votisfiod py t^gft compOEOtsve t^ffo^f^nro i^fiCjfv ot-^ mi' 
low comporotfve peff<vfnor^re on NAf P t»ut gfven tf>e j^jtcj <>r> tr^^ st<.-v<3./,g c.f tr*^* 
U"(ted StQtes tn >ntr2»rTX3tK?n?3f stadte? of stvjder.r porfrrmofve i fvivt* iik^wnM su>. h 
feefjfxjs otx>ut o slo'e s fek>tive fiofff^fnan*^ p o'> ttx» MAf P to ttgnt tn^j for ff »e .of 
toiAer decH on the tftonsc wtvt>f^ the ttptx^tgs fjre in stgr>t T^keIe s^rr^.^i^* o^(^ toi 
trnpcviont fSiues tho* fn^ utw^fHt of us os o r>otio^ sixn rv>w v^e ossu'e rno» pvO'^ 
srudent wtx> ^eoves ^«o^ s^t^oo* fs wo*^k modv tn *hf> g*ot ''i jc^ir^t.tve f^^o'Ver vvo '-^ -J 
o^-Ksetves »n 

l?ote thc^ NciUonof Auetsmenf Governing Socsd » r./ivt» o.;e<K}^ Kxk t ^ j cv> t»ns v >• 
r> my ro<f^O'kS OfSOwhOff. i t»nvt> fvod tv^th p^js'lfwe nnri rrf^go^'we tcos-f^g^ ot« tt**-- 
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Boofd ilrco inception Th© posirrvo orm <Jv© to H traic»v r^^p^e50n1o1twe ncy»uf€* 
or<Jit>kTd©p©fxJof>c€>f»omTn©controctof Tnts tbsoev© Ha LseM feature o^w^ 
BcKyd whfc^ ou5^rt K) t» »©taJf^ OtkJ expGr«Je<i ijpcri ^ t have cfiSO come 

fo tee* o f ^\je clefiife on the port <^ naGB fo* sfoto tnptrt on vott^ issues foc^ Not 
Assessment 

On the negative sJoe I tjefteve ttKJf tt« Sooki ^«35 not o^wcjy* sougni out o<M:e ?fom 
proctltlonefi before OC^tng & not $uffk:»onttV staffed m ordei to carry out some 'ts 
resportsftj^ttes (sucn as ttondofd settr^). is pressured tnto actfrig Defore shoi^o 
(n 5cx7>e rr>et»ure to unretfSticctfV short ttmefnes) onc3 r>asrti oSiws teer> owore o* 
NAEPs past nor gven consKJerotw to txTw tf co^ er* {of reptoce) of^>ef teoefopy 
funcjed dcio coeecrw efforts oo student pt oduft ocrsevernent 

Mowevw. i nove cc»T»e to re<^i?e tftot o moK^f port erf these p*ot>Jerr« is cfue to not hovmg 
petsons vyt)0 <ye famrto? wim the state- ieve» eoucotionoJ cy technjcoJ tssues SL^ovnvJ^ng 
assessment on tt^ Govetrtng 6oafd <t»eif i feet that a mo)Of cnonQe to the ao<yd wcx;ui 
be to Odd two o^ more <»sessment directors ond cumcuMrv defectors tJorn stotesto tt>e 
Governing Boord A!so CorryTess shouMS require ttxst o toctvticdi odwscry pwp 
COrtn»sed of vorious coretftuent groups, ifKUOinQ state and kxroJ testing directory tx» 
esto^esTied ond that siicfi on odvisofy group De feguioftv c(X«suJted n the esfot)tehr^cni 
of aswssment poScv ond proctice t>ef0fe the Board a>aJ«es t»yj« doctstons on etthor 
wnce s^andinQ commrtteet txjve Peen used for short pertods of hme and aflLDcs, 
groups twe dso been med the on-going cxJv>ce wh»ch o stondrig group coutd 
pfovtdo tx» been m(Si*ng Both of ttwse ore CnttCOJ needs wtMcn shov^O be toker^ co^o 
0* m tt>e »^ OUttvytKjtiorj of NiAGS 



Summory 

J tvjpo that my romortcs ore ne^fU to you OS yOo consoer the W>p<> ofvj scope the 
Natonoi A5se5sn>ent of Educofttono* Progress of the future As o supporter <^ the 
pfogrom ft>ope that my rernarks w« be heipt^^ to yoy OS you dtscuss these ond othe* 
important issues There Js o useful ond construdtwe ro»e wtvch o Mationoi Assesir* v>i 
conpiaybut corefu!thooght.portici*aityby tr>osefomiiicrviftfht^ -r* 
of the past as weO os tf^ proct>ca> ond tectvvcai issues sunour<Jing ossessment wt6 
help to ossure that tr© post ts not r04nvented no* thot use^v^ ottemottve osaessments 
designs ond strotegtes ore not considered My cotteogues ond t $far>d reody to t»tp 
cyx5\*/«t»ew<^<ng tobecoisedon We believe thot we hove rrKtchuseftrf to contribute 
ond ask thot wo beco^ne port of the process of dficussion ond decision on tnoso ond 
otfter importont tssues 
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MICHIGAN DEPARTMENT OP EDUCATION 



MEMORANDUM 



tO: 



FROM 



DATi: 



SUBJECT: .St<it*> ffTv-^VOfr-Pn! in thf> 1 »f»!,iya {>f Nfjf.nnal Aw>5&fnOf.t 

Since my mtt^n f'Oin WtnJj^figtt^n f ) C »\n M<virtoy ntghf, t hnv^ gfv#vi rf.>n^«'Jf»ior^o 

tnvjil^rtKi tn on pfXJSA? of tfw> (jf»^*gn anU ronfi»Kr» thb |i»c^'^t From fec^»nt^ at, 
<»fj»jeCft*onal ono conceptual ponts of vfew, ouf f»xportpncc»S 0i9 tnvtjfiX3l?i?» »f not 
tj/"il<V«» toi agf>rKif>V swkkb>Q 1(7 tlt>vgn a nf.3ti/>n<?f ff>sflnQ pfogtom wNcti f*VKtv^ down 
t<"» t^e Slot^ Of k> nl k»vr^ It wos thH vif»vrf whic h mf» io s\iQpost of! n>e A; >»iJ J9<?0 
f ctucatky»rr>fonT>oft*in Afiv*vvy ff>mrFvtJ*>«»(f tAD rTtfv»f.r>vj thcif fxirttccKjfif jn »f i ftu> 
NAf P Sf<3t«» ASSf>55n^nt puigttwn t»n ronltuyorrf f>r»<iBf»< f Sf<Tt« if^p^ inio t^)f* ri<j»Mif»fjl 

Af thp ou1Sf>J, t wonf to c k^fif y f^Kil i Si)tH>vinr of Nf i«tOf>C7t AWi-smont As n fof'TWi 

Uj-iW*! by th© momef in wtilr h sffif** fruxjt not srKjqhl ^Jftyr^Ki lf> tho rJf>s«\:;n f >f rjAF P$ 
UfiiUfi »wTotf»otJO«Jt»fiHooJon»;orv7? 1TOorx*Hv»5#n^«v3rt the $rtiJoticr>fvrt not 
C^VKvQ#»{jf ftrtdormnffjBy sinro thpn ffv* fptKiJory 7f> moot fnfl o% wof? us ibA o< fJvit.PS of 
of retwojy ?6. w«B «f<rt1ff:rtf>s ffK> fnprf* Bp smvirn ftiot afv>*>of$ io Q^w^n to 
m^orrinofii stote Inptif ^fo tf?p Nof ofxi? A$so«smonf c^stpn Wtilifk ft^e most t»wK)J 
os|-^?S of NAFP om itwen (yi thP loAj^m ffrHjr* orvl <tc»s*gn wo^k f nu^n. f ,f 
wJfbout states hav*ny o mf>on^>gFiJ oiitxirtufitty fo irorrtfft^^nt 

At the ff^bfucnv 25 moetjixj of:tM!tPs tniattriQ to tt\? Wt f>eW t<*sl of ihe |v<Tgfo:n 
woie cJ^5cu»ed. os wetf os futrth*>r rifoft* of ttv> tC?';»o f^^pofH w«»fe tviixJed out anu 
d«5Cus«KJ at rm>gfh Vfituoily fh^? Pnl,rp mo^nng was c?<*vot©<1 to tti© pu^i^' on of 
descrtptivo motertoJ from Wf si AT N< itS Of CCSSO ?b'$ materto! was prp«*ntA<i to ff^o 
largo QftXJp In ci {»c!lir© fnrmot OfXJ «<tte cv no Of jjxirfunrty wo< mo?^ <7^^ilot>M> to 
octtjaBy hcv© Inpvt from th© vo^-fous testing qvkS cufrtci/fiim pprvons fn cJttf»ndon<.e 
WtTfta tt Is cflfPJctjft to hvrTW fn f^rtpt? » <i.v- u\sif jn w.!h ! pf>opk» o» o mfkpting »f ts not 
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tnjM>sslWo Given th© numb©* cTf Nc:f WF STAT or>d \ TS sToff ;:^5(>nt ft wouki ^V7v^» 
IXKKi ou<<0 pO»Jbk> to bftKJk the torg^^ group No flfloen to fwf>fitv smoO^w dKciw^on 
grovjps ond th© stoff <3S recordpi? I cofcrutofed thot the tax pcw cosf for th© 
me©ttnQ wos otx>fO»^^oV 5>eiOXX» For fhft c»7>Oiinf, 1 quwlloo wh©tf«w any 
«)oriso*tng OQ&ncv Of <3ttencJ©« mc«lvod any Wor mottan thot wOuH couw* cjny ooo to 
do onvthir>g d*Tof©ntV fven the li*r>o*hy npw drofts tho ISiAEP imports ^« cTv^^ohJa 
only on ?t« momino rrKHJtlr^ so thcrt rirporfunm<*« few mcM on the«> wojo tirr^ftwi 
by th© voujm© of TnaHGtioi to be ^evtowed 

tn tT>o rrwntlme. o vc^ry Impcvlcffit est ijsues ?o b© focused on at the Tuesday puhfic 
hecplng on the 1994 and l<79$TSlAfP wos passed out CoovonientV. notrJcclion otXJUt fho 
puWIc fwjr^ woi rocefvsd by irt |us? o few doys bef?^ th© meeting. weU cf tPf trovo! 
rior« were s»f ft was cipof to me !h<jt core ogo^n. 8p s^^n.-ice wos being ghw to slate 
lnvolvi>m€»nl Into dmign or N AFP WNi© yoo hod vMi;olV NAEP Coofdincrtors ond 
mothen^oflcs u/peM^of? in tn^ Unlt^ Stot*»s in Wosh^on D C ond codd hove ecnay 
devotcxJ on tio^it to the topk:. porticuioity using smoQ grovg:) dtecusstons. no ptovHton 
wos modB for us to c^iiss the Hwes worse itrfJ now of the meeting porltipcjnts hod 
o ctxyK:e to frame orirSHonaJ issues or options ttKJf cot^d be pfosentea ond tfecussed 
fho next dtiy crui thtete ore sevefc^ which, foi wt^otev^ r©<3Son were orrxJItcHl from the 
Is! One such Isst/e li th© rk©ed Ujt o study Ot mottvatton of twelfth gradors on Notkx^oJ 
Assessment tf the tirrrfng wos bot). th© Jon^tay AssoctotJon of StoJe Assessment 
Program s/Et AC Assessment 5ut>commjtt»e rrw»tlng cotikf have been uspd Ut f) not«ty 
Slates ttv3t o pubac hfxr/tng wtn to bo heW and 7) ctov tnpul on the tssues optKxn nst 
As o momtWf of ihfl Cor^resslonriV outtvirtred NAiP EvatiMJtlon PQnel- 1 find rt 
Interest^ thot trm Evo^JOtion Pqnol olso (Ski not have on opportuntty to comFTV>nT on a 
odd to tt>e Bst of tssAm even ffyxign we hod o meettng fus! a few weeks ogo 

imimotpty. ' Washington very fri«tfated and onpry obout how my tlm© hocf bp*»n 
obiiscKf. as wPB as complet^V imceiloin < - *^th©f NCFS or CCSSO pkv\ Ut LfJta?P tf>f* 
FIAC Auessment SLtfxrommtttee in ccJe* to got systematic Jnput on th«*se ond ofhef 
Issues mkited to no^KXXTt tests cn6 e'^omtnotions M seerr^ to me thoJ ft should be 
po&slb^ to design ways to rere^v© (r^>Lft ftc^ the key pJoyers £n moortngfi/! ways m a 
ttmofy monnef wftho< rf viotat*ng government pf orectures op coi a*ng us to tokp^ eve" 
moie th>e owov cxff K^bS t*oif1ing mcwtmgs at ronsidorcd>te expense to simpty 
pfos^ motortat wtiich cotfld b» put m writing is oof constructVe 

In ctosfr^. f request thot NC£S feconsWer the monneF fr^ which stoles ore ottowed to haw* 
Input on the r^*onot Assessment pfofect f^. th© ^^AEf* ^te^varts meetings s/>oufd 
isovtde more tfme to hove :nput on NAfP. of o time and in o monnef whkrh oiJows ax.h 
hpirf to ofTecf fhe design of the pfofert One way communJcorfion <5tx-h o? has 
chc^octerlfod these meettngs) can best tie hondied *n memorondo w^^ch we can »p<i<i 
of home UfSfee the meetfrigs to »eoBy t^eor from the stores Second, when opportun«»ps 
nrtse to 'dteuss' newty devetoped Hems, such os fhe fwo pfOPOJed stote comporfson 
mock ups. pfo»rfrte opportunft^es in tt>e meeting fo» octooi smoS g»oi;p discu^^ion of the 
ftpms The piesontaffcjn ond 'discussion' of th© items m o foge g^'Ot^p lecture fomvit 
Shook) no* be disguised os a tfue opportunity fo« Input on this pfojec* 

nnntty whon mo}or flosign and ronr**pttK38?ot loo lisues €r»ls« such <t^ wf*ie to 
cflscussed for ttw 1094 and tC9s6 P on Fehfu* ry 26, why not let tf>e flAC A«c«mf»nt 
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issue*? Torne.thbbfh© Yivoh^mont'fhaf Imeortf hAf^M990on<Jii^ 

ftxri the new Ass©ssnr«nf SubcommTHee formo! woiAJ oSow UnforfiP^f©?<. who? I hovp 

S^jen ts That |tot©» ctb ofv©n minJmat opportunWIos tor Input ond f h<m on onfy the most 

lk>k^ !h($ change*. I sIK fo«l siForjgV ttcrtw should not portic^te Jn ni© 1997 
program FfonkJy f om stS not Si^e wtM>Jho» Mk-hipon ihoUd be o portk ?jx7rTt in 1992 i 
hopo ttxil (ho «loto5 wS express Th©^ foolngs otxnit the cJeiign o? th© Na?K)noi 
Assessment prefect caxi tho woyi ^ wtifch stcttoi ore gNfon cjppoftunrftfes to hptp shop?* 
ft I urge ynu to reconsldef the woy$ In whferh ttatos ore e^kTwed to hcjve ir^xjt Into this 
pjolecf WTitle YOU moy be comrrrftted to Iho FIAC ABOttmenf Subcommtftoe 
opprooch for ttite. ore you to trtOfre tt>e vOfv nrtechonlsm you h<ve e$»<7blW>od to 
aSkjw mch Ir^>ut7 Tbe <tesK?n of the t994 an<j NoTionof Assessments pfov*i<>d rhf» 
best oppoftumty io fo? to test yw/r commitment flow tnie tnvoVpmenT of tN« siovs 
in franrrfng the iss;;es on<J responding fo fhem b** coftted ou*? 



cc Assoc {oHon of Stoto Assossmonf Pn>QTom$ 

Assoctaftton of fJtntn S^s^onrHf^^^s of Mrrthpmrrtt.- ^ 

DovkJ Ekjytess 

Steve Koffler 

Gory rrofSfn 

f?omsoY SPlttof! 

Rotwwto Sfnnfoy 

Ikjrnnno tfij^lott 
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Chairman Kildek. Thank you very much, Doctor, for you*- testi- 
mony. I am going to pick up something you alluded to at the end 
and throw the question out to all of you for our enlightenment 
here. 

Testing for the m<st takes place at the end of an experience 
or the end of a process. What do we discover in testing that can 
help us to improve that experience or that process? Testmg is gen- 
eraJly an omega point. What do we do wiih alpha, beta, delta, 
gamma— if I have got my Greek alphabet right— what do we do at 
the b^inning of ♦he process? How can we make testing helpful in 
the improvement of education? 

I think that is probably Congr^'s greatest concern, to improve 
education. How can testing, not in isolation be helpful in the im- 
provement of education? 

Dr. Walbkrg. I would like to take first crack at your question, 
sir. I think the implication of your question is extremely construc- 
tive, and I think we have to think of testing at various points in 
the educational process. 

For example, Dr. Roeber mentioned the consensus projects that 
were carried out by the chief State school officers under contract 
vith the National Awe^ment (Soveming Board, and this is a way 
that educators can ^t together to formulate the test to say, "What 
are the very b^ curricula to offer, the best instructional prac- 
tice?" Then these can be translated into test items. 

Then when the test is given, very specific information can be 
3rielded by the National Assessment in terms of where students in 
a particular State have done well and where they have done 
poorly. 

For example, common fractions versus dec mal fractions, and 
this gives very specific curriculum information to people in State 
departmentr. of educaition or we have national organizations such 
as the National Science Teachers' Araociation that can look at the 
strengths and weaknesses of various curricula areas, and that way 
that there is a constant feedback or interaction between the educa- 
tion process itself and the testing which should jointly enrich one 
another. 

Chairman Kildee. Has anyone else got a comment on that? 

Dr, Linn. Very briefly. I think one of the ways happens all the 
time in an individual classroom. A good teacher is doing the kind 
of testing all the time throughout their instruction, and it really 
becomes merged with the instruction. That is very different than 
the kind of tMting we are talking about when we say a national 
test or even the National Assessment of Educational Progress. 

Where the two sometimes get linked is in what is happening in 
some States right now where groups of teachers are working to- 
gether on the type of assessment, and it is really kind of staff de- 
velopment type of work, coming to agree on the kind of perform- 
ances they are trying to encourage, how they assess those, and how 
that relates to their instructional program. 

Again, however, this is not a mandated national test but rather 
one that is closer to the classroom. 

Chairman Kiu)ee. Dr. Roeber? 

Dr. Roeber. Yes. I think that the testing that would go on- there 
are different levels of it. There is a cla^room level. There is a lot 
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of informal testing that Dr. Linn alluded to. There is also at the 
State level in our State, at least, assessments that go on that are 
built on frameworks that are not unlike the National Assessment. 
But built into that is a lot of asistance down to the classroom 
teacher to provide them with strategies; for exaiiiple, to get the 
teachers together in grades one through four and say, "We have 
tested these students in fourth grade. Where should they have 
learned these things, and how do we help them, group like these 
students, the next time around and then what do we do with these 
students that have the needs?" 

The National Assessment could do some of that. It is very diffi* 
cult to have that, but I think we first have to have the attitude 
that that is appropriate, and when I su^ested that, for example, 
the National Council of Teachers of Mathematics or the Mathemat- 
ical Science Education Board be given the rraponsibility of develop- 
ing strat^es like that, it was indicated to me by staff of the Gov- 
erning Board that that was not an appropriate role for a Federal 
agency, and that is why I made the comment that it is okay to test 
people and make them look bad, but not to help them. 

It also means that moneys have to be available to groups so that 
they, in fact, can do that. I don*t believe that commissioning the 
National Council of Teachers of Mathematics to say, "What are the 
implications of this assessment framework for instruction?" means 
that you automatically end up with a Federal curriculum, and I 
think that is the concern. 

The area of mathematics is one in which the framework is some- 
what forward-looking, and one of the major things that the results 
this summer when they are reported, I hope, will focus on is that 
this country really needs to reexamine the whole notion of mathe- 
matics instruction. 

We have treated it as if it is a special talent that a very few of us 
have, that we are born with, and the rest of us, it is okay. There is 
no other area like mathematics. There is no other subject area that 
you will get grown adults that are presumably well educated that 
will laugh with one another about l^ing illiterate in this area, and 
so that one of the fundamental changes that has to occur in mathe- 
matics is for us as a nation to say, "It is not acceptable to have 
adults who are weH educated who are mathematically illiterate,*' 
and it was this notion that I wanted to try and use the June 6 
NAEP release to try and get to. 

We are going to have public attention about schools, and one of 
the things that we need to focus public attention on is the ne^ to 
change the attitude about mathematics. All students and alt ages 
need it, and it is not a special talent that only a few of us have. It 
is something that we all can develop, and the nature of mathemat- 
ics is changing, so that rather than using the NAEP results re- 
leased on June 6 as an occasion to say, "Schools have failed," we 
ought to be using it as an <K*casion to say, ''Here is what needs to 
be done to change and improve mathematics instruction,*' and I am 
afraid that that message will not be the one that is featured. 

Chairman Kild££. Go ahead. 

Dr. KoRETZ. Mr. Chairman, if I can take a somewhat different 
tack, I think your question is a very good one because a number of 
the proposals that are now being floated at the national level 
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would, in fact, take a test that is given at or near the end of high 
school and use that as a mechanism for driving ^ucational change 
all the way down through the system. 

I think that approach is very risky. It assumes without, I think, 
a whole lot of evidence that somehow the incentives, the new in- 
centives, ii you will pardon the metaphor, will trickle down to the 
younger ^^es, and that, therefore, we don*t have to worr>' too 
much about how that happens, 

I am not sure that is true. If you go into a third grade classroom 
and look at mathematics instruction, for example, the issues that 
the third grade teacher should be facing are things like, "To what 
extent should I get rid of traditional drill and replace it with the 
use of manipulatives? To what d^ree should I replace individual 
seatwork with group problem solving?" and any number of ques- 
tions of that sort. 

It is hard to see, in my mind, how a test given in grade twelve 
will manage to inform those decisions very well unless our goals 
for education are so narrow that they can be specified in a very 
lock step sort of way, and in my experience going into classrooms 
and watching how teachers have responded to State testing pven 
every several years as opposed to once, there is some trickle down 
there, but it isn't necessarily what you want. It sometimes com^ 
out in the (ocm of pr^ure for the most routine and boring sorts of 
drill. 

I think the real question, which is oflten unanswered in the na- 
tional testrig propc^als, is just what is going to happen inside 
schools to change the incentives that teachers and students face? 
Just simply having a test at the end of the road isn't necessary 
going to do it, I think. 

Cluurman Kildee. I apprdciate your answer because I think that 
is rather fundamental to this issue. The key qu^ion is how can we 
use the tool of ai^essment or testing, to improve education. That is 
very important. 

I ask that because— and I am going to personalize this a bit. I 
taught Latin for ten years at the high school level, and I can recall 
my first couple of years, I just couldn't understand why my stu- 
dents a>uld not master what we call the sequence ot tenses, when 
jrou use the indicative mood and subjunctive mood and must deter- 
mine which tense you should use. It was always very clear to me 
because I had spo^ Latin for six years in college. That was the 
medium of instruction^ and I just couldn't understand why my stu- 
d^ts couldn't master those ^uenoe oi tenses. 

But through testing I discovered they just really did not imder- 
stand it very well so 1 redid the way I presented the material in a 
very iMical way, and I discovered that my slowest student was 
then able to understand. 

So that taught me, then, that by testing mv students, I was able 
to find out they weren^t doing well, werent understanding that 
well and therefore I could change my method of presenting and 
discovered the ri^t method* 

If we can link education and asseesmmt in some way with the 
improvement (tf education, then that makes us more conucMrtabie in 
feeling th^ it is going to adiieve some real purpose. 
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Very often in education we start at the Z point rather than the 
A point. I think even in schools of choice, which is now on the Hit 
Parade, that we go to School Z but don't ask what we ought to do 
with School A which the students are vacating because they don't 
like what is there. I think we very often have to start at the ban- 
ning. We can t^t, find out what we can do to improve education 
there, but then apply that in some way. 

Do, for example, the National A^ociation of Teachers of Mathe- 
matics, when they get t<^ether, look at tests and say, '*What can 
we do to improve the method of delivery of mathematics?" Does 
anyone want to comment on that? 

Dr. Walberg. Well, I can say that in general the national profes- 
sional organizations such as the National Science T'^achers Asso- 
ciation, the NCTM— that is National Council of Teachers of Mathe- 
matics—are very test conscious because they realize that there is a 
danger in the test driving everything, and that, I think, is one of 
the reasons why testing is so salient these days. So many of the 
professional ^oups at the national level are looking at th^ thin^ 
and translatmg them through various types of consensus proce- 
dures, but it is also being done at the local and the State level, too, 
and just as you said, and Dr. Linn mentioned, a teacher might give 
a test and then see if that, as you illustrated, is helping the in- 
struction. 

There is a deliberate attempt to link improvement with assess- 
ment, and I think it is highly constructive to do that, and that is 
why I would say that the National Assessment Governing Board 
would like to make itself as ui^ful as it possibly can. We have 
many demands; there are many things that we are being asked to 
do, and I think that many of them are very valid and legitimate, 
and we have to figure out which kinds of things that we could pos- 
sibly do best. 

For example, we just heard, I think, two {x)ints of view. One is 
that we should give a lot of attention to the— that is, the National 
Assessment Governing Board attempting to directly improve educa- 
tion, and giving prescriptions on what to be done. 

Another group would say, **No, just do the monitoring alone. 
Let's just have the facts, and leave it up to local educators and 
people in States to do that.*' 

So I think we have to be very careful about what our purview is, 
and I think the board hai tried to consider that very deliberately. 

Mr. Martinez. Mr. Chairman, would you yield on that point? 

Chairman Kildee. I would be glad to yield. 

Mr. Martinez. You know, the idea that you are talking about, 
testing, conjures up in my mind a couple of problems that I have 
always seen as my kids went through school and as I went through 
school. 

In testing, most tests that I have seen to date actually seem to 
test how much knowledge you have gained, what you remember 
about what you have learned in that class. It has nothing to do 
with how well you will perform any place else. Who is from the 
RAND Corporation? And you sav scores can improve dramatically 
because you teach to the test. I know that very well because there 
are real estate cla^s that they give. All you learn* all you do in 
those real estate classes is memorize the questions they will prob- 
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ably ask, and memorize the answers. That is what a lot of gaining 
knowledge in the classroom is, memorizing facts, hoping that that 
is what the teacher will ask the question on or that is what you 
will be t^^ted on. 

But ass^ment to me means something very different. It means 
asse^ng how well that person will perform in that subject matter 
r^ardless of what the test might show, and I don't know how you 
do that exactly, but there must be a way of doing it. 

I mean, I would think that in a case where you ask a question 
with only one answer, you ask the student in a test to i^rform a 
certain task. Write a busing letter, you know, rather than ask 
him, "Is this busing letter constructed accurately?" 

If you are going to do aaessment, I would think that assessment 
keys to that ability to help because you assess where the weakness 
is and then you target that weakn^ by extra instructions or extra 
help. In the case of the national ability to help, you asked the ques- 
tion, "You can beat up on them because their t^t %ores are not 
good, but where do you help them?" 

I would imagine the Federal Government, which doesn't want to 
get involved in curricula, could provide moneys that would go to 
schools that have identified a number of students with weakness or 
that need extra computer help or materials or whatever it is, or 
even specialized teacher help, to improve that particular situation. 

So my question basically is, how do you distinguish tetween test- 
ing and assessment and how do you achieve what the Chairman 
has asked for, the ability to assist? And where is the greater ability 
to assist: after assessment or after testing? I happen to believe it is 
after assessment because that is where you identify the weakness- 
es. 

Dr. KoBETZ. Can I take the first stab at that? There really are a 
couple of qu^tions there, Congressman Martinez. One of them is 
how assessment or testing can be used to actually give teachers 
and administrators information to improve instruction. 

I think the answer to that depends on the context, anu one of the 
things that makes me nervous about test based accountability — it 
made me nervous in the 1980s when States were doing »t; it makes 
me more nervous now when people are suggesting the federal Gov- 
ernment do it— is that it is very hard to Know, sitting miles and 
miles away from a classroom, what information a teacher needs. 

There is sort of an implicit model in some accountability models 
that teachers really know what to do but they are withholding, and 
if we just set the test at the end and give them incentives, they will 
finally start producing for us. 

I think that is not true. To reduce this to an anecdote, I recently 
went into a third grade classroom. It was the highest track. It was 
a school that tracked even in the third grade; highest track math 
froup in a school that scores at the top of the scale, basically, on a 
State mandated math test, but the class was having clear problems, 
and I sat down with one student and simply asked him to describe 
why he was doing certain things that he was doing, and the answer 
was he had no idea, none. 

Since then I have heard parents in that classroom starting to 
complain that their kids are doing very well but don*t know what 
they are doing. 
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In a sense what happened is that the State mandated test sup- 
planted—it didn't add to, it supplanted other kinds of information 
that the teacher should have been using to find out whether kids 
were learning and what they needed to learn. That kind of dis- 
placement, I think, can occur quite often. Deborah Meier, who is 
the principal at Central Park E^t High &hool in New York, said 
that there once was a day when if parents came in and said, "How 
does my kid read?" the teacher would say, "Let me show you what 
kinds of things your kid can read, and let me show you some things 
that your kid can't read/' Now they pull out t^t scor^. 

So I think that there is a danger that centrally imposed testing, 
while it can sometimes have, I think, very desirable efTects, can 
also undermine precisely what we are trying to improve, and that 
is why as we move into — apparently we are moving into more na- 
tionalized testing, I want to see more attention to what mechanism 
is going to be set up to guarantee that this information actually 
helps rather than hinders. 

Very briefly, one answer to another part of your question is, I 
think there is now a very widespread desire to increase the content 
and the range of content of what are included in tests, both at the 
national and local level, and to include things that are much more 
along the lines of writing a letter rather than correcting it. 

So I think at least on that — we don't always know how to do that 
in terms of rigorous assessment, but there is clearly a movement 
afoot to try to do that so I think at least that part of the problem 
may get better. 

Qiairman Kildee. Thank vou. Dr. Koretz. Dr. Roeber? 

Dr. RoEBSR. Yes. I wanted to comment on that because it really 
touched on several of the — your question, I kind of touched or* in 
my comments, and let me elaborate a little bit. 

I mentioned that performance assessment used to be a key fea- 
ture of the National Assessment program. From 1969 to t'.e mid 
19708, in fact, when I worked at National Assessment we were basi- 
cally given free rein to draign assessments that hsst assessed the 
subject area so in career and occupational development, if we 
wanted to know whether people had life salable skills, we asked 
them, and if they said they could wait a table, we went to a restau- 
rant and said, "Show us.'' 

In the area of science we gave them complex problems such as 
Congress is wrestling with such as "How should we generate elec- 
tricity in the United States?" And then we scored the depth of 
their thought and the kinds of reasons that they gave. 

In citizenship the extent to which they were taking thot^htful 
stands on complex issues without correct answers. Many of the 
things that we most value and would, in our own workiti^ defini- 
lions of what a well educated person is. National Awessxnent was 
asscysing. They did it within a reasonable buc^t on an axmual 
testing cycle in the early l^Os. There is not a lengthy institutional 
mem(»pir, however, and so now we are reinventing this and calling 
it untried and unproven and going slow on it 

ThB aeocmd point is that the value of the vmmg adult assess* 
mmi that I refened to was that it forced the demff^m of the as- 
wwmen t to think about Bnmkmdj idio has gone thrcm^ all of 
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their formal schooling. What is it that they should have? What 
should be the residual of that? 

And what that really got us to thinking about is, well, we go 
through school and we learn all those school skills, and you can 
test those, but what are the real pnxiucts of that? How should I be 
able to apply that when I am 29 years old, which I wish I was. You 
know, how can I apply that; at home, on the job, as a citizen, and 
all of the roles I would play in society. That is far different than 
thinking about life as made up of mathematics and science and 
writing skills. It really talks about the application of those in vari- 
ous life rol^. 

And finally, I think if you base your assessments on those then 
the types of assistance that you provide to schools begin to make 
sense, rather than forcing them into drill and practice for memori- 
zation, in fact, what you are setting standards that would cause 
them to move away from that and talk about is their instruction 
leading to good learning that stays with people at least as late as 
age 35. 

What NAEP found from that young adult assessments is that a 
lot of the school stuff that lead to memorization work is clearly for- 
gotten by adults, but those life application skills that ITyear olds 
were very weak on, adults fortunately are very strong. They prob- 
ably learned that through the school of hard knocks, unfortunate- 
ly- 
One of the reasons why I would like to see that reinstituted with 

the National Asse^ment ih I think it could provide some guidance 
to our high schools or secondary schools as they look at that t\ere 
is more to life than going on to college, and that even students who 
go on to college go on and get jol^ and have to fulfill a number of 
life roles, and I think that that would give us a broader perspective 
on assessment and hopefully a broader perspective on instruction. 
Chairman Kiloee. Mr. Goodling. 

Mr. Goodling. Thank you, Mr. Chairman. I guess I would ask 
you to respond to whether mv feara are justified or not. I realize 
that we are partially responsible in the Federal level that some of 
these ideas that seem to be driving the wagon at the present time 
because we are always talking about accountability and quality 
and excellence. Now the governors and the President are talking 
about standards and how you measure them and achieve the na- 
tional goals. 

I guess the question I have primarily is, are we doing these 
things, whether it is choice, whether it is testing, whether it is 
teacher training, whether it is national curriculum, are we doing 
them because we don't know what else to do and we are just sort of 
throwing up our hands and saying, *i don't know what to do." On 
my side of the aisle, particularly, I think probably nine-tenths of 
them graduateu from a public school, and I think they were suc- 
cessful before the> came here. But if you hear them talk, public 
schools must be the worst institutions in the world. 

So I guess my concern is, are we just throwing up our hands 
when we come up with the idea of maybe a national curriculum, 
maybe national testing, maybe choice, et cetera, et cetera, rather 
than clearly thinking through how these different moves may 
better our education in this country? 
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I heard Al Shanker, for instance, re<^ntiy say that he believes 
there are a lot of good possibilities in relationship to choLj if it is 
not used simply brcause you say, ''Well, I don't know what else to 
do and therefore apparently choice is the answer to everytMng/^ 

I keep trying to nnd out what the administration means, for in- 
stance, when thev talk about choice. I haven't found that out yet^ 
but perhaps I will be lucky enough one of th^ days. 

Does anybody want to respond to my concern? Is it justified or 
isn't it? 

Dr. Linn. I will respond, particularly in the context of using tests 
in this way rather than choice. I think that too often that is exact- 
ly the case. Tests that have been at State levels, at district levels, 
and now being discussed at the national level too often have been 
viewed as a cheap fix, as the engine of reform, as opposed to more 
systemic changes in the educational programs. 

One difference that we now have in a proposal for a national ex- 
amination syBtem as opp(^ed to a national test of the type that is 
being discussed by Lauren Resnik and Marc Tucker is the notion 
that you have to stait with what the syllabus and with the edu- 
cational prc^am and curriculum itself, and ^nat the exam follows 
from that as opposed to leading it. 

Congressman Kildee's earlier comment about his Latin class, I 
think one of the implicit points that was there was that you had 
clearly in your mind what the Latin was that you wanted the stu- 
dents to be able to demonstrate, and unl^ you have that, the tests 
can lead you astray and in the wrong direction from above. 

Dr. Walberg. I would like to add to Dr. Linn's points* I think, 
again, trying to take the National Assessment Governing Board's 
point of view, I think our goal has been to provide the best infor- 
mation we possibly can and to be somewhat r^rved about giving 
prescriptions. 

We heard Dr. Roeber advocating in a sense more prescriptive in- 
formation, telling what to do, and we heard Dr. Koretz expressing 
grave r^ervations about that. 

I think the Board's view is that we would like to do the very best 
we can to provide the information, but what to do with that infor- 
mation might have certain causal uncertainties about it, and each 
State may be different, each school district, and so what we would 
like to do is give this information and let local schools, States, Con- 
gress, and other agencies draw implications for policy. 

We have not been given that specific charge to do that, but I 
think if we sort of stick to our last, that is one of our best func- 
tions, and I think that that is the Board s point of view. 

Dr. Koretz. Congressman, whether or not your fears are justified 
I certainly share them. I think a number of the major policy initia- 
tives that are gaining credence today are of the sort that we final- 
ly—people are hoping that we finally found some lever that we can 
pull from afar that is going to make the whole system fall into line- 

I dont think there is such a lever. If you will pardon an analogy, 
it is in some sense analogous to the idea of a vaccination. We are 
looking for some kind of a vaccination that we can find that will 
work. If we do, then the government can say, "All right. We will 
fund development and use of this vaccine, and we are all done.*' 
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I think a tetter analogy for what we are facing is the efforts that 
have been under way with Federal support for several decades now 
to reduce heart dise<^ and cancer where the answer turned out to 
be, unfortunately^ there is no vaccine^ and there is no antibiotic. 
What we have to do is somehow arduously do the work to get 
people to change their lifestyle, to live healUiier lives and to avoid 
some of the risks, things that would put them at risk. 

In many ways improving elementary education and secondary 
education, I think, is analc^ous to that. We have to change a lot of 
people's behaviors and expectations all through a very fragmented 
and large system, and I don't think we can do it with any one 
lever. I think some of these levers can help, but I think the attrac- 
tion of them, and the reason why they are in many cases oversold, 
is because it gives us something we can clearly do from one place, 
from Washington, from a State capitol. We can just do it, and hoj» 
that we can wash our hands of the matter. 

I think if we had more realistic expectations for what some of 
these centralized initiatives could accomplish, we might well ac- 
complish a lot with them, though they wouldn't be the whole 
answer. 

Dr. RoEBER. Well, I think that we do tend to put a lot of faith in 
tests and hope, perhaps, as well. One of the things that I have seen 
from the years of experience that I have had in Michigan, and cer- 
tainly Pennsylvania has had as many yeara of testing at the State 
level as we have there, we tend to view tests as, first of all, a way 
of determining whether or not there is a problem, perhaps as a mo- 
tivator for change. We embarrass enough people or we can maybe 
be more helpful and then just embarrass them, but then also want 
to rely on the same test, perhaps, to evaluate the effectiveness of 
the change, and I think we are asking a lot of a test. 

We began in the early 19708 what really has become a national 
phenomenon because we had some researchers in Michigan that 
were very concerned that testing was being used to indicate that 
students really could not profit from schools, that all that they 
could learn was predetermined, almost, before they came there* 
and the State began a whole train of research on effective schools 
and also on the prwess of change. 

If you know the characteristics of an effective school, one that is 
serving well, minority students and othei^ of low economic circum- 
stances, can we go about taking that information into schools that 
are similar to thvt but where the f^rformance is lower, and what 
that research indicated is that the classroom teacher has to be ac* 
tively engaged in the proce^ of change. 

So that assessment in our loosely coupled system— and I say 
loosely coupled because wa have a national test which may or may 
not affect State policies. We have State assessments which may or 
may not affect \(Ka\ policies. We have local assessments which may 
or may not affect school policies, and school policies which may or 
may not affect the classroom teacher. 

In that kind of loosely coupled system to expect that a test from 
the national, the State, or even the district level, is going to pro- 
foundlv affect classroom teachers, may be unrealistic, but on the 
other hand if we look at the change process as starting with teach- 
ers, that they have to be actively engaged in a school improvement 
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process that they feel a stake in, then I think that there is a con- 
structive role that tests can play. 

Using the health analogy, just finding out that the average 
Americanos cholesterol is too hih won't necessarily motivate me to 
change my eating habits, but if somehow somebodv can get to me, 
then we can wonc on that process of change, and I think that is 
why I see that it is critical that whenever we test, we try and pro- 
vide people with information that thev can use to improve. That 
isn't making them do it; that is not holding them responsible that, 
"You not going to keep your iob imless you do that." It is to 
provide resources to people and the encouragement because it is 
very, very difficult to affect classroom teacher behavior, very diffi- 
cult. 

Mr, GooDUNG. I guess one of the concerns that I have, a.'^d 
maybe I was incorrect in doing it, but I alwaj^ tried to convince 
teachers that the m^or purpose for their exam was to deterrnne 
where they failed to present the material well, whatever j vas 
they wanted the student to learn, and where individual students 
did poorly and what they might do to help that student become a 
better student. Then when I near people talk in terms of national 
tests, it kind of reminds me of my first experience as a guidance 
counselor. 

I wasn't asked whether I thought grouping in seventh g^ade 
would be a good idea. I was told bv the superintendent that I would 
'TTOup seventh grade students and so I took the sixth grade records 
and I eventually decided that this one teacher— they must already 
be grouped in sixth grade because the one class was just outstand- 
ing. Everyone in there did very, very well in all the results that I 
had so I went to the superintendent and said, '*Do you have them 
grouped already in sixth grade?" He said, "No, why do you ask?*' 
and I told him, and he said, "Well, you have to understand Mrs, 
So-and-So always teaches to the t^t." 

Well, you know, it sort of took care of my grouping. I thfjik you 
very much. It has been very rewarding and very meaningful, the 
testimony that you have given. 

Chairman Kildee. Thank you, Mr. Goodling. Mr. Gunderson? 

Mr Gunderson. No qu^ions. 

Chairman Kildee. Let me ask some more questions here, then. 
Dr. Roeber, we have had 22 years of experience in Michigan in test- 
ing. Has the testing process in Michigan been formally utilized to 
try to improve education, getting back to my first question. Is there 
a linkage there or has it just been reportorial? 

Dr. Roeber. No, there has. We have very actively engaged the 
Michigan Council of Teachers of Math, the Michigan Reading Asso- 
ciation. They helped us ^tablish the frameworks on which we base 
our assessments. They help us generate the assessment materials 
them^lves so that they are as content valid as they can be, arid 
most importantly, once we get done with the assessment* they alito 
help us in interpreting those scores to teachers. 

We place a great emphasis, perhaps unique among even State 
programs, in using the assessments to have a positive impact on in- 
struction. I think if we set the right targets for our students, I am 
less concerned about teaching to the test or teaching to the assess- 
ment If we are testing the right things in the riglit ways— I don't 
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mean — I don't want people to practice the actual test items, but I 
want to make sure that they are teaching the right things, and 
that is why we have to have that involvement. 

It has led, for example, in the area of reading for the reading 
teachers in Michigan to really consider that that reading assess- 
ment is theirst ana although the results are low, that is because we 
have set some fairly high standards for our students* and as a con- 
sequence people feel quite, quite desirous of helping schools im- 
prove their instructional programs in reading. 

The reading people without State resources ha^^ put on literally 
hundreds or thousands of workshops around the State for class- 
room teachers suggesting how the model on which the assessment 
is based can and should afTect instruction so that the asse^ment, 
when it was implemented after the five years of development, 
found a very receptive audience because the teachers had already 
learned about the instructional model that was related to that defi- 
nition of reading, and we are eagerly k^king forward to the first 
assessment data so that they can see how their students are doing. 

Chairman Kildee. You suggest. Dr. Roeter, that a conceptual or 
statistical link to international programs of asse^ment could drive 
NAEP upwards. Could you go into that further, and how do Michi- 
gan students — you mentioned you are not too concerned about how 
they comi^*^ with Indiana but mayl^ internationally — how do 
Michigan stu%lents compare on ass^smenl with their counterparts 
in the fourth, eighth, and twelfth grade who go to work in the 
Nissan plant in Japan? 

Dr. RoEBER. I wish I had data for that. In fact, an opportunity 
was provided to States. Unfortunately, it was too late for us in 
Michigan to take advantage of it. It occurred afler our budget cycle 
was completed, and although I had pledges of private support, that 
was made to a previous governor, and for political changes was 
withdrawn. 

Chairman Kildee. I recall that evening the change took place. 

Dr, RoEBER. And as a consequence we couldn't gather the data. 
What we were proposing to do was to select a Michigan representa- 
tive sample, augment the lEA data collection, and at the same 
time administer to that sample a portion of the Michigan reading 
test so that then we would have been able to predict scores on the 
lEA scale from the performance of students on the Michigan sc^Ie. 

The linking of those two scal^, then, would have allowed us in 
fature years when the Michigan test was given for us to come up 
with predicted lEA values, and that would have allowed us to do 
that. Unfortunately, the funding was not available in time when I 
had to essentially pull the plug. 

Ironically, the current governor is making a lot of noises about 
wanting to know how Michigan students do in comparison to stu- 
dents in other countries so we may have a chance to fund this pro- 
gram next year. As I understand, over 30 States have indicated an 
interest in this. 

What that would have allowed us to do is to report the propor- 
tion of students at a basic or proficient level in this particular 
Michigan school building in the district, in the State, as w<*il as in 
the 37 countries that are participating so that I would have gotten 
the national data equivalent to what I would have gotten out of the 
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National Assessment Project, but in addition I would have gotten 
the information from the 36 o^er countries. 

These kinds of things are possible when there is a similar enough 
definition of the subject matter to be tested and when you have 
given both t^ts to at least a sample of students so that then you 
can make that statistical link. The same sorts of methodolc^ could 
be used with National Assessment. 

There is an alternative that has been done, and that is to take 
National Assessment items and administer those internationally. I 
am personally not in favor of that because all we are doing is 
seeing how other countrira do on American standards. The real 
value of the international program is that we see how American 
students do on international standards, and I think sometimes our 
standards are different. 

In the area of reading, for example, the U.S, committee on which 
I serve was predicting that the United States standards in reading 
would be higher than those in many other developed countries. 
Our students may not perform as well as th<^ coimtries, but at 
least we are expecting more and want more out of our students 
than these other countri^. Wo will have data, probably by this 
summer, to see whether or not that prediction was correct. 

Chairman Kildee. Thank you. Dr. Walbei^, would you elaborate 
for the committee on ways that comparisons of States or districts 
and of schools using N A£^ results will help improve education? 

Dr. Walberg. Well, this is a controversial qu^tion, but the 
Board has had a lot of interest on the part of States and local com- 
munities that wish to know how they stand on National Assras- 
ment standards^ and I think because of the release of State com- 
parisons this spring and our performanw standards that we will be 
reporting on in the fall, there will be even more interest in this, 
and I would say educators are very interested in knowing how they 
stand on a NA£P scale, but we should not forget that it is not just 
educators involved. It is State legislators, business people. I think it 
is quite clear to many Americans thrae days that even on our own 
tests, that is, NAEP tests, when they are given in other countries, 
we have not done well, and so there is a tremendous national con- 
cern about our standing. 

So if they have to wait for lEA results which may only be given 
every four yeai's internationally^ or they have to wait a long time 
for NAEP results, they may institute certain changes in their local 
school or their State or we may institute changes in the Nation, 
and if we have to wait too long to get the information, then they 
have to make decisions in the meantime. 

So by various ways it would be possible, such as in the old days, 
to use what were sometime called released items, that is, items 
that had already been used and therefore wouIdn^t hurt to use a 
second time. That is the fact that it wouldn't hurt the National As- 
sessment itself l^(^use thc^ items could be used by local districts. 

It would be possible to either reuse them themselves or calibrate 
it to local tests so that a test in a particular school or district could 
be calibrated to the NA£P and at any time that they wished, in 
June or August, they could give a test, calibrate it with the Nation- 
al Assessment, and see where they stand. 
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One of the great dilemmas in my opinion, sir, of testing is that 
we don't have a meter stick that is kept in Washington or formerly 
in Paris where you could compare any student at any time with a 
universal standard. 

NAEIP does have that possibility, and I think if we moved in that 
direction, we would have to very cautious because we have to 
protect the trend lines and we have to protect the security of the 
test, and we also have to maintain professional standards such as 
not reporting individual students' information. 

I thmk it would have to be done very carefully, but I think the 
testing, if it was wisely and ea)nomicalIy done, efficiently done, has 
the potential in some of the ways that vou have pointed out in 
giving information on how to improve scIimIs, could be made much 
more useful tcxiay by having these kinds of calibrations so that 
local school districts and Stat^ could make much more use of 
these things and on a very efficient basis, too, so that we would 
avoid too much testing so that it enhances rather than interferes 
with the actual education program. 

Chairman Kild£E. Yes. 

Dr. KoRETz, Could I offer an addendum to that? I think different 
tests could help in different ways, but I think the National Asse^ 
ment has one possible way of helping that nothing else does which 
is it can just simply tell us whether things are getting better or 
worse. 

If you look at the wave of State level education reforms in the 
1980s we really don't have as much solid information as we ought 
to have about whether that extraordi/iary burst of reform did 
much, in part because when States use '*ests to drive instructional 
systems, they often didn't bother to put in place another, uncor- 
rupted test to see if anything actually got better. They used the 
test they were using to drive instruction to monitor change, and of 
course scores went up. 

But we do have one sign about what happened in the 1980s, and 
that is the National Assessment which nobody bothered to teach to, 
and it didn't budge. That is actually an overstatement. It budged. 
There was trivial improvement in some areas, slight improvement 
in others. 

So the NAEP managed to tell us, not with as much detail as we 
would like and not as well as we could get out of a formal program 
evaluation, but it gives us a pretty good sign that that wave of ac- 
tivity in the 1980s didn't do what we ho|^ it would do, and it is 
the only thing we have in the United States that can tell us that. 

I think as the next wave of reform tak^ hold, it would be a 
shame to lose the ability to monitor whether or not we are getting 
better which is currently only offered by the NAEP. 

Chairman Kildee. Yes, Dr. Roeber? 

Dn Roeber. Dan referred to the trends during the 19808. I would 
reference also the trends during the 1970s. There was substantial 
improvement in NAEP f^rformance during the 1970s at a time 
when our nation — perhaps not the States but at least the Nation— 
paid considerable attention to the achievement of poor and minori- 
ty students in the United States. 

There was both statistical, and most importantly educationally 
significant gains, on the parts of these students. Unfortunately, 
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while the gap nanx)we(l during the 1970s, that gap did not continue 
to narrow during the 19809^ and in addition one of the trends is 
that the performance of students at the highest levels of the NAEP 
scales, the percentage of students that were capable of performing 
at that level, has not changed much over that period of time and 
continues to be quite low so that I think that also illustrates, 
again~I would agrra that it illustrates the value of a monitoring 
system. 

You may not be able to attribute it to a particular policy that 
took place in a particular State or school district, but yon can see 
whether or not things have changed, and I think that the 1970s 
was a time when we paid a considerable amount of attention to the 
achievement of those grouf^, and we saw significant changes in 
their performance. 

I think that was most heartening to see, and as a reviewer of the 
20 year trend in National Assessment that was something that I 
particularly paid attention to in my comments on that 

Chairman KiU)£E. Mr. Goodling. 

Mr. GooDUNG. Just one last question. A lot of people seem to be- 
lieve that imless we change teacher training institutions we are 
not going to change very much. Is there a role in NAEP, for in- 
stance, and testing pn^ams that m^ht help us encourage these 
changes if you think changes are necessary? 

And I guess a follow-up to that wuuld be are the teachers trained 
presently in the use of the results they get from the testing that we 
now do, or is it just something that we put on the shelf and say it 
was nice. 

Dr. RoEBER. It is very tempting, Mr. Goodling, to suggest that if 
we simplv changed the training of teachera, that somehow that 
magic bullet would also change student achievement. I think that 
is clearly a piece of the answer but not the only one. 

I mean, most of the teachers that are teaching now will be for 

auite awhile and so we have to worry also about whether or not 
liose teachers that are currently teaching have the skills that they 
need. 

TTiere are a couple of ways in which I can answer, ''No, they 
don't/' First of all, the teachers in most States in the United States 
are not required to have courses in assessment or measurement to 
become a teacher. I haven't done a national survey on that, but 
there have been some regional surveys. I know in Aiichigan they 
are not, and I have yet to find a State in which they are. 

That is probably good because if the^ did get a course, it would 
be the wrong one. What we need to do is not t^ch teachers how to 
build testa. I mean, there are testing technicians that can do that« 
and they clearly need to be trained in that. 

What teachers need to know is how to link asse^ment results to 
good instruction. That m^ns a formal test, be it a NAEP or a 
State or commercial test or district te^ or whatever, but it really 
means, more than that» the kinds of informal testing that Or. Lim 
referred to, and teachers have not been trained in that 

The second thing that tih^ are not being trained in is the ways 
in which the national cunriculum groups are redefining the content 
areas. Scienoe* the science area» has been significantly redefined in 
two veiy diffident w^ya* cme by tiie Nat»mai Sck»iee T^ichers' Asr 
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sociation and the other by the American Association for the Ad- 
vancement of Science. 

The typical classroom teacher of science, particularly at the ele- 
mentary level where you are a teacher of everything, those wajre of 
reform have rolled r^t on past them. The same thing is happen- 
ing in mathematics, in reading, in writing and other areas so that 
while the changes, those changes need to be reflected in teacher 
trainings those are future oriented actions. 

What States are wrestling with now is how do we work with 
the — in the case of Michigan— the 88,000 teachers that are teach- 
ing now, most of whom will be teaching five or ten years from now. 

We have to try and provide as they are working, stru^ling, to 
teach these things to students, we have to try and teach them the 
assessment methods that they can use, and we also need to teach 
them how to incoriK)rate these major curricular changes in their 
instruction; obviously a significant challenge for our State and 
local school systems. 

Chairman Kildee. Yes, Dr. Linn? 

Dr. Linn. I would like to just elaborate on one of the i>oints, and 
that in the context of the National Council of Teacher of Mathe- 
matics curriculum standards. 

They provide a different view of what a^essment would be like 
than is too often modelled by tests and so I think that having a 
better modelling of what that is, when Congr^man Martinez was 
here he referred to the notion that you might learn all those facts 
and do well on the tests, but not really be able to solve the prob- 
lem. 

Well, that is a good illustration of the type of performance as- 
sessment that is trying to be developed now and modelled for 
teachers that would in mathematics not just have the notion that 
there is a right answer or how to add two numbers te^etb^^r or use 
a particular algorithm to always get a particular answer, but 
really, often it is figuring out v*hat the problem is and approaching 
it from difterent points of view and coming up with multiple solu- 
tions. That is more important to mathematical thinking than 
having a factual answer. 

Mr. GooDUNG. Responding to that last comment, 1 use the 
phrase so often that "He has all the answers, but he dc^n*t under- 
stand the question/* 

Dr. Walberg. I would like to add to what has been said, Mr, 
Goodling, by saying that there has been something of a national 
debate atx)ut the content of teacher education programs, and in my 
view it is very unsettled. 

There is one view that our teachei-s need more fundamental sub- 
ject matter background. If you are going to teach physics, you 
should have more physics courses. On the other hand, there are 
those that believe that you need more courses in pedagc^ which 
would include testing so I think it is rather unsettled, and many 
universiti^ are still debating this question. 

Some of the critics of American education have said that our fun- 
damental technolc^ in education has remained the same since the 
turn of the century. It is basically teacher presentation, recitation, 
board work, seat work, homework, and despite all the facts of all 
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the new innovations with computers and media and other tech- 
niques that really aren't being used very much. 

That may sugrost, and I Uiink we have to admit, that testing is a 
fairly arcane subjwjt. It is not somethii^ liiat— people get Ph.D/s 
in it, and it takes a long time. It is very statistical, maUiematical, 
and requires a fair amount of specialization. 

So to some extent it rould be aiigued that if we want to look at 
new innovations in education, that there oi^ht to be testing ex- 
perts or people who would help the teachers in the school, that 
would specialize in converting information Uiat they may get from 
NAEP scales. State scales, locally developed instrumentations, and 
that this would be hooked up to computers so that both teachers- 
well, let's say teachers and parents but i^rticulaily students them- 
selvet^ would get instantaneous information and fcN^back as to how 
well they were doing. 

I don't view this as a panacea. I don't think it is proven, but I 
think it is something that ought to be given a try- Testing could 
play a very valuable role there because many teacher made tests 
are amateurishly done. They are not as well done as a technically 
reliable test. 

If we could combine these two to make the test both rigorous in 
the scientific sense and relevant to the teacher and the student, it 
may be a way that we could increase American educational produc- 
tivity. 

Chairman Kildee. Thank you. Dr. Koretz, you make a statement 
that NAEP is so complex that NAGB needs additional technical ex- 
pertise to make appropriate decisions. Could you give illustration, 
or would you call for restructuring of NAGB? 

Dr. Koretz. Well, let me answer the second question first. I 
think whether NAGB needs to be restructured depends in part on 
what decisions >ou make in the reauthorization about what its au- 
thority should be. 

I think there is a lot to be said for the current structure in that 
educational goals are not a matter, just of prof^ional expertise. 
The public ought to have a strong voice, and so having a National 
Assessment Groveming Board that represents diverse public views 
and diverse constituencies, it seems to me, is very desirable for one 
end of the decisionmaking process. 

But there are, unfortunately, a lot of very difficult technical 
questions that come up. There are, for example, a lot of technical 
problems with the current effort to set achievement levels, stand- 
ards of achievement. There is, to go back to performance assess- 
ment, there is a problem in deciding when an assessment Is suffi- 
ciently different from what we have done already that we need to 
investigate how it is done before we use it for national reporting. 

NAGB does have a resolution of its own suggesting that when 
changes are really dramatic, there should be two trials before a 
new assessment is used for national reporting?, but even the 
changes that have been made in the national version of the Nation- 
al Assessment to date, in mathematics, for example, raise some 
very serious technical problems, and there really isn't a mecha- 
nism now for resolving whether the decisions that have been made 
are the right ones. 
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The pressure is there to simply produce those assessments in the 
quickest possible time i^cause people want them now. 

For instance, at a recent meeting of the Technical Advisory 
Board that the Educational Testing Service has put t<^ether to 
adv^ it on the conduct of the NAEP, several of the members of 
the advirory committee responded to a presentation about one of 
the new assessments by saying, ''Well, do you, in fact, have any in- 
formation that this new set of items and new set of tasks actually 
tells us anything reliable?" 

And the answer was really, **No, we don't have much." There is 
no pl«^ce in the schedule, and there are no resources set aside for 
finding out whether that is a good way to do it or whether we 
should be doing it some other way instead. We simply have to get 
it out in the field. 

Ideally, it seems to me the National Assessment Governing 
Board, if it is going tn have responsibility over technical decisions 
of that sort, ought to have the ability to decide when they are 
g^ing far enough in that direction that they need to, say, hold the 
line against rapid change or call in some Kind of technical review, 
and I think over the next couple of years as we get further into 
performance assessment we are going to have more and more of 
thc^se issues coming up. 

Chairman Kildee. Dr. Linn, you stated that the current trial 
State assessment is not sufficient to make a determination of the 
success of full scale assessments for grades four, eight, and twelve. 
When do you believe there will be sufficient data to come to a 
sound conclusion? 

Dr. Linn. Well, that depends upon what Congress decides for 
1994, 1 am afraid. 

The point that I tried to make is that by 1992 only a part of the 
system will have l^n tested, and in particular trial at twelfth 
grade will not have been undertaken at that point so I think that if 
the interest is in eventually having twelfth grade included on a 
State by State basis that it would prudent to have a trial at that 
level tefore it is instituted operationally. That was the m^jor point. 

Chairman Kildee. Thank you. You answered my question. That 
is what I was trying to discover. Thank you verv much. Dr. Linn. 

Does anyone have any concluding statements? Ves, sir? 

Dr. Walberg. I would just like to mention one point in response 
to Dr. Koretz about including other technical experts on the Board. 

I think that that merits consideration, but I would like to point 
out that we do have two full t'nie psychometrists on the staff. 
NCES has various experts that they have assembled to advise on 
the National Assessment. The National Academy of Education, a 
committee co-chaired by Dr. Linn, is giving us evaluation and 
advice on these matters, and we have felt free to assemble techni- 
cal panels on highly technical issues to make recommendations to 
the Board. 

Also, it has been very helpful to us to have the Council of the 
Chief State School Officers has carried out some of the technical 
work such as the consensus panels, and they have done a lot of this 
work for us, I think, in a very useful way. 

1 almost think that right now we have so much technical advice 
that it is almc^t too much, and we need a broader perspective that 
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you would get from a more publicly constituted group, although we 
are certainly seeking all the information and advice that we can. 
Chairman Kildeb. Dr. Roeber? 

Dr. Roeber. I would like to comment on that because that was 
one of my remarks, opening remarks. 

That is, I believe strongly that the heart of the expertise in the 
practical matter of carrying out, developing, and using assessments 
belongs in the States, and the State people have been badly under 
utilized in the development of NAEP. Yes, there have been a 
number of ad hoc groups^ advisory panels, and so forth, and in fact 
I think that that is one of the dangers that I try and avoid in run* 
ning my program. I trv and seek out a consistent set of advisors 
that span both the policy, curriculum, and assessment expertise 
areas so that all of that is brought together in one place, and in 
fact people can inform one another of that. 

For example, in the policy discussions about the length of the as- 
sessment cycle, I am not sure that there is anyone with the exper- 
tise to talk about what it takes in the way of time and effort and 
procedures to adeauately develop assessments. The asi^ssments 
that are being develo{^ at the national level take anywhere from 
a quarter to a sixth of the time that a State would allocate to the 
same process, and the procedures nationally are much more com- 
plex. 

I think by having people with that practical experience on the 
Board, then when the board is setting policy that can happen. 

Second of all, 1 think there should be a standing advisory com- 
mittee that can deal with advice to the Board from a technical 
nature. They have come and they have gone. We were wrestling— I 
was appointed to, I believe, the second of those two panels. At the 
start of the second meeting we were dismissed, and the Federal Ad- 
visory Committee Act was the reason why we were dismi^^. 

I don't believe that that procedure should stand in the way of the 
Board getting ongoing technical and operational advice from the 
individuals that are most qualified to provide that, and so as you 
consider the role of the Groverning Board for the future, I urge that 
you carve out a place in which tne people that have the practical 
experience of running programs and developing frameworks can 
get actively involved in this. 

1 think it is just a good strategy for the Board, anyway. Thanks. 

Chairman Kildee. Yes. 

Dr. KoRETZ. If I could quickly add to that, I think the notion of 
having a standing panel of advisors as opposed to ad hoc arrange- 
ments would be very desirable. 

In fact, in my full testimony that I submitted, I suggested that as 
one alternative way of getting expertise to the B<mrd. That not 
only brings expertise to the Board but gets people with technical 
expertise t^ether in a room to discuss £ind a^>ate issues that are 
in front of the National Assessment which I think will improve the 
quality of the advice thev give to the Board. 

Chairman Kildee. Unless there are further statements, I want to 
thank all the witnesses who testified. I really believe that we have 
a^embied, really, one of the greatest panels I have had the privi- 
I^e of listening to. One nice thing about auditing a course is that 
you don't have to take a final test. 
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[Laughter.] 

Chairman Kildee. Your testimony has c»dded a great deal to this 
subcommittee's understanding of NAEP and the educational test- 
ing issue. You raised a number of important questions* I would like 
to submit, if you would, please, additional qu^tions to each one of 
you for the record. 

In addition to that we will keep the record open for two weeks 
for other submissions, and again I thank yois and with that the 
subcommittee will stand adjourned. 

[Whereupon, at 10:55 a.m., the subcommittee was adjourned sub- 
ject to the call of the Chair ] 

[Additional material submitted for the record follows ] 
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QUESTIONS FOH DR. ROEBER 

QUESTIONS 

1} YOU STATS IH rOHR TSSTI>(Oliy THAT A COIfCBPTDAL OR 
STATISTICAL LINK TO INTERNATIONAL PROGRAMS OP 
ASSESSMENT COULD DRIVE NASP UPWARDS. WOULD YOU PLEASE 
ELABORATE ON THIS STATEMENT? 

2) SONS EDUCATIONAL EXPERTS BELIEVE THAT NASP CANNOT BE 
BOTH AN ACCOUNTABILITY TOOL AND A INSTRUCTIONAL TOOL. 
THEY SUCC8ST THAT NABP CAN EITHER BE USED TO TELL 
POLICYMAKERS WHAT KIDS KNOW OR IT CAN BB USED AS AN 
INSTRUCTIONAL AID TO HELP TEACHERS AND RIDS IMPROVE. 
CAN NASP OR ANY TEST DO BOTH? 

3) DO WB RUN THE RISK OF OVERBURDENING OUR STUDENTS AND 
TEACHERS WITH TESTING? HOW MUCH TIME IS ALREADY 
CONSUMED BY TESTING IN THE AVERAGE STUDENT'S ACADEMIC 
YEAR? HOW MUCH MORE TIME MIGHT BB TAKEN UP WITH 
TESTING IP WE MOVED TO A SYSTEM OP NATIONAL ASSESSMENT 
OP ALL STUDENTS? 

4) TO WHAT EXTENT DO WE ALREADY HAVE "NATIONAL TESTS." 
THROUGH THE MAJOR COMMERCIALLY PREPARED ACHIEVEMENT 
TESTS THAT ARB ADMINISTERED IN SCHOOLS THROUGHOUT THE 
NATION; THE COLLEGE ENTRANCE EXAMINATION SUCH AS THE 
SCHOLASTIC ACHIEVSMBNT TEST AND TUB AMERICAN COLLEGE 
TBSTING PROGRAM, OR THE FEDERALLY FUNDED NATIONAL 
ASSESSMENT OF EDUCATION PROGRESS? IS THERE SUFFICIENT 
COffSEfiSVS ON WHAT CHILDREN SHOULD KNOW TO CREATE A 
VALID TEST OR SYSTEM OF ASSESSMENT? 

5) IN YOUR OPI3IION, IS THE USB OP NAEP AS A "THERMOMETER- 
TO MEi'iSURB THE STATUS OF U.S. EDUCATION COMPATIBLE WITH 

* AN SXPANDBD USE OF NABP AS A HIGH STAKES NATIONAL 

BXAHINATION COMPARING PEPPORMANCS OP INDIVIDUAL SCHOOLS 
OR INDIVIDUAL STUDENTS? 



1 



c 



41-090 0 - 91 - 



94 




DEPARIWENT OF EDUCATION 



SIATt WOMm » fDUCAIiOii 



imnft Mi ghigg i «B909 




MQiCh2) t99t 



»AlJf Oauiiw 
• MM It* 




fr>e Honorable L)aie £ KKieo 
Chcaiman SUjcommitte© on fl^T^eritory 
SecoocSary ond Vocofiono' FduCOtton 
Commffteo on €<3ucotion orx3 laDo 
u S House of j^eprescmtottvies 
33D CcsfffXJn House Office Buac$ng 
Wos^w^gJon DC 205^5 

Thonk yow TO» youf Wnd romorto f&gat0tr\Q my te$f tfnony to the Sudcommrfie© or tn© 
OOCJmontf OtJestfon* on off ochmen? 

> txjpe t^^at these commonts w§i t» h^ipM »o you orvj ttie off>©f Suocommfttee 
m©^t>e*s »f t^y Of stoff^>(^feo^vQu©stIoasabo^t^5, please t©^ 
□oo^^cff SI 7/373 83*3 TNar* yoo ogoin foi tt^ opportontty ot s^<sf^^ 
pe^^Oec^^ve on MA£P 
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Modi sum 

T^© D^^pose tn fostng 'tis posslWihr of 

Test A and T©u B txjm covot ftw tc»t>o content c»«o (e g rrocfeig) ^ o smraKy monney 
If v^&opo*«>»topr0C*cticore$onoo©te«stfromtt>eoJ^ Thu «fat*rt»cal r»»«ng ts 
(XCcyncrfJshedtwNTvinottwsan^setofttuctentf ta^ Th^t^© 
scofWtTorn the tv«j tests con be p»oc«OOott«iomescc<e in The future, orty lost A w 
Tosi BUrf«olbomr»eGdbeo*v«aTheK<»Wontt>©onetiwf not ghf^con^ 
pf0<*ct©<S from the one tixjt 

Conc0pturf te^kng is coneo oJf When ttid content of two tests ore so srr^ ttxfl tn^ two 
exoms oi^ essentK^ mposuftng some tiv^ thtt roroV happens so ttvot s»ot<sK<^ 
bnkr>g « oimo$i oN»K7v» needed 

2. CaiNAlPb«uaMtboma»ar>occouniabetvtooiandoniRStnicno^ 

The dtemmo <s foced o8 tns ttm© a* tne ^ote ond Jocca tovel f?aT«*v or© t«fs given 
w^MC^ we do not try to ^ ©ducdtofs to d»ect»v lee !o*npiov© teoct^tng ond Jocmng 
At ttx» natKjna* Jeve^ Not*onai Assessment * ttw pes? such exonw that we hove erf or» 
oisessment not cunwrty des»oned to cfirectiy tmpoct msifuctton of to ho*d stote a txai 
educoto<» accountot?*e tpf perfofmorice 

ft HASP scores Of© used to pjOqb state or kxai 5ctxx)( systems maEP as 
accountot>*tv too* wHJ De eohonced Howevw the nwe o t^ ts used to promote 
occouf^ObiWy . the i©»it5 v<^fof fTSfnjctkJ^ Bducoto*^ moy Decome 

5o concetned otXH/t snxtent peifomxyice tt>or ^nsteod ijs*>0 trie ossessment as of^ 
mOKTO^o^ The need to review ond tmprave the insmx:t»onaJ pfogrom. they moy simpv 
^eoc^ ?he ojoos of jtudent weotmass without pfow»dih9 the cwropnoto ntegioted and 
tong tgMm rvjtruciton wt^ch student* teotty need n ofdef to fetotfi the concepts 

)t IS not jmpci*sC>ie fof o test to se^vo oott^ purposes tus^ tne mofe 0* one piifpose does 
tt>e ie« of t?>e othef thct moy be feoabie 

3 Am school o vrtx^der^ wgtt too much fsflng ond witt new te^li odd to tha 

r>«fcie«7 

5 do not trunk ycxi coiAJ ftfxJ many educotort Who Defcev© tnot the*© « too tttie tostir^g 
One mcto* feoson ^o» ttT« i suspect s that much extefnoi testtn© «s not tied strongty tc* 
instructcn Hence, o^y ^h dote will be viewed OS eiftrone<Xf$on<^ not needed 
Anothe* ossessment wiQ simp*y odd more testtnp trr* to the buf oen tvyj ft ur&Wety to 
ctKmoe educoto>$ optnsons obout testrtg 
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Tho orrro\jr\t ot ttme <y» tosfir*o vonas from dfttrict to d»Jnc1 an<3 stot© io Stcrto os 
w©f OS from stucjeni to ftwdent Those i*udonn who of© r^vo^ved r, specoi pfogronni 
<» e «?oc»a) eOKtcdon ot con\peniotoiv ©<Jucaton) ^eceNo e«tfo tests on top of those 
CSf^o^jyrootfneiy given TO OS students It woUd noJ t» tiaprtsnp to find ttxjt somo 
stu<i©nt» witi spen<* npwf«» erf twisnty how* tena yoorty on t^r» 

New not*onoi exonranotKKB depewximg thoir notLffe, ctHid otld os attie as on hout cv 
twototT^etottf.ofnsomeof therno«o*atoofoteicheme« might f^u»« w©o*s cv 
montf-« of prepOfoton time a$ weii m constdorobie t*m© fo eoch fiuOen? to co^ty ou^ 
trve# performonce 

4 Do w« ^rwocty hov« 'n^onoT Mti? te ttiM foBidM cofVMfmis <m chU^en 
tf«ouitf Know l» ^ MItf o mHiDnal twit 

The cr-twef to m»* & tX3fh orxJ 'no * V©s. therD or© vomxa tests whch ore grvof . 
natxxxUfv bol fX? tfjese not gfvon to oP rudenfs at o common grcKJe level cv of© noi 
gfvon to resprosGntotfve J<ynpj© of students ipm© coses, such as the ACT and SAT 
they c»e t«cen onty Pv a seff-setecTod somp^ o» students »o that the ooto ts no* 
repfesenfoiive of the state Of the dWrtct tn the cdse of tt>e comn>erctaBy ovaffobtp 
to$fs senoui <^j©stKx>s hofve oeen pottoo /i&oofdfng tt\e i^esenfoffweT>ess o* fr>o 
nofms 9fOup6, smce tfuj « o voiwntcyy oo*»on on tne port of ttie dwtrarti and jcfxx^i^ 
seiiKrted m the sompie 

NAtP >s one o^ fhe notton<S^r fepresentotivo ossossn^ent* wh^ch or© gve/i ri the U 5 
Other* nclutto trie och»©v©nr\ent sechon of tt^e ^rfot^onol Educolioool longrtudtnol SttjOy 
<^4lS)ondtt^e <ntemdTK)nd»£ducotx>no* Achievement <JE A) $tud©» wow^ve* the^iA^P 
osM»ssment » the most co*T^>J©t>ensiv© of those assessment progroms 

5 do pebeve that there 9 (or con bo) sufficient consensus on wtxsi our students need to 
know fh ofde^ to cteote o voba sysJem of csse^smoni tn foe? { wouKS worry mote otKXit 
vo^OiA stoles OfH3 toco! dtstncts setting dfffefont syitoms m ptoce Mowevef . the 
feosipji^tY ond desi*oP*i<tY of orrvsTg at o consensus of who* we wont our stud<*n»i 10 
hnow does not meoA mot nottonof tesfs ore needed Of deswed 

5. ts uting NAEP o»q ^tfifm o me li ' co<iyoKba» wtm »»pQndfa^o NAEP tnto o high tto*<es 
noHoAOt #JKvn<noflon for ichoots or shi^er^? 

TniS <$ ymMof to questw two Do we expeci o tt>e*mome*et to heat the wotei rf ft fuKJS 
the temperoturt) too co^d'' Does fi sttfi fetlect occufotety wtxjt T?>e woter temperotuto 
wos (or ts)'> Would another thermomete* ftfxJ the woTef tempefdture unchonged'' Ono 
dangoT in tho jomo test to fYKKssute watet ten^pefotute ond to *heot the watet 0$ 
n^entloned m the response to Ques*on two ts thot ttie chonges *n student pertormofK <i 
may Pe more ottnputoLe to pfesss^ put on sctx>Ote try the pOft<uicr test cmo 
fep^e^ent true sttjoent teorntrig vvhtch woukj gef>efOH?e to ott>ef compofOPJe rneosut*-s 
the comparat>io tests fa uwhKin schools ore not teochtrtg mov not show tn© so'^*^* 
^Cfovement Henre tne of the thefnr>on>ete« nr>ov be destroyed Dv uSi'X) ^ ^<-> 
'heot the woter * 
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COMMITTEE ON EDUCATION AND tAuOR 



ITS HOUSI Of RfPf^SCltTAriVlS 





MB • rvA>V *vM«p arc 



WA&HttfGfOH DC 209 IS 




AWO VOCAnOMM tt>UCAT.ON 



Itorch 15. 1991 



K«rb«rt J tfalb«rg. Board tte^r 

1100 L SCr««t. NV Sulc* 7>2? 
UAstngton. Q C 20005 

D««r Dr tf«lb«rg 

TliwOi you wy Buch for th« Mc^lX^nt t©»cl»ony you presenteiS mx. the 
(Urch 13 » 1991 hearing of ch» Subcoaittee on Eleaentory, SecofxJary, 
and VocAtionat E<luc«tlon Tlw Subcoaattttt r*cotv«d a «M«lch of 
valuAhl* Inforutlon vhith mill greatly »»»4»t ftur «fforc« to revUw 
thm ItAtloMl AM«sM*nt of Educacioit Progr»«. 

Enclosed plM«« find th« additional quostlons vhich I indicated 1 
would be making you t» retpond to for t*»« record Due to the great 
Interoit in the ieaue. I would appreciate your eetlstance In returning 
the eniwered Rations by March 2?. 1991 »0 the hearing racord can be 
printed quickly. 

Again, thenk you for your velueble lestiBonj? end your eealatance In 
expediting the return yoor anat«ers to th» additional quest lonA 

Sincerely. 



Dale E Kildre 
Cheirsan 
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encloeure 
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gUESTZONS FOR OR* ItfAL&ERC 

QUESTIONS 

1) XN YOUR OPXHION, IS THE USB OP NABP AS h * THBKMOMBTBR ** 
TO MBASURB THB STATUS OP U.S. BDUCATIOH CONPATXBLB WITH 
AN BXPANOBD USB OP HASP AS A HIOH STAKES NATIONAL 
BXANXNATION COMPARING PBRPORHANCg QP INDIVIDUAL SCHOOLS 
OR INDIVIDUAL STUDENTS? 

2) DOBS THB NATIONAL ASSBSSNBNT GOVERNING BOARD SEBK MORS 
THAN THE AUTHORITY TO DO PLANNING POR STATS-LEVEL 
ASSSSSHBNTS BEYOND 1992? GIVEN THE ABSENCE OP ANY 
EVALUATION OP THB STATE-LEVEL ASSBSSKBNT EFFORT TO 
DATE, IS THERE ANY REASON FOR THB CONGRESS, AT THIS 
TIKE, TO CONSIDER PERMANENTLY AUTHORISING STATE 
ASSESSMENTS? 

3) PLEASE DESCRIBE THB PROCESS THROUGH NKICH THE NATIONAL 
ASSESSMENT GOVERNING BOARD SET ITS BASIC, PROFICIENT, 
AND ADVANCE THRESHOLDS POR PERFORMANCE ON THE MATH HASP 
ASSESSMENT. HO» DID THE GOVERNING BOAPD MEASURE THE 
RELATIVE DIFFICULTY OF INDIVIDUAL QUESTIONS ON THB NAEP 
ASSESSMENT? HOW SHOULD THESE THRESHOLDS BE INTERPRETED 
AND USED BY POLICYMAKERS? 

4> PLEASE DESCRIBE THB CONSENSUS PROCESS WHICH NACB GOES 
THROUGH TO DBTBRNINS TEST SPECIFICATIONS. 

5) WHAT RESEARCH EVIDENCE IS THERE THAT FORMAL STUDENT 

ASSESSMENTS AND THEIR RESULTS ACTUALLY LEAD TO CHANGES 
IN THB BEHAVIOR OP STUDENTS AND TEACHERS? ARE THOSE 
CHANGES PRODUCTIVE OR ARE THEY SOMETIMES 
COUNTERPRODUCTIVE? 



o 
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Or. Walb«r9 



Qu«atton« 



P»9» Two 



7) 



8) 



ERIC 



HAS NAGB on nkSP 8STXMATKD WHAT IT WOULD COST FOA A 
VATXONAL TEST GIVER TO BACH CHILD? 

HXTH THE INCREASED ATTEHTIOH AT THE STATS AHD LOCAL 
LEVEL 2H SO->CALLEO AOTHBf^TIC ASSBSSNSHT, THAT IS, TESTS 
or WHAT STUDEETS ACTUALLY RJfOW AHD CAH DO, TO WHAT 
EXTBMT HAS HASP IKCORPORATBD ART OF THESE RgW 
ASSBS8HBNT TECHRIQ0BS7 BOW DO KINORITT AHD 
DISADVAHTAOE CHILDRBH PERFORH OH AUTHSKTiC ASSSSSHRNT? 

PLEASE DESCRIBE THE PROCESS NACB HAS ESTABLISHED WHICH 
BRSUR88 THAT QOESTIOKS ARE BIAS PRBB? 

HOW DOES LACE OP NOTIVATIOH APPECT THE ACHIEVENEHT 
LEVEL OP STDDEHTS, PARTXCOLARLT HXOB SCHOOL SEHXORS7 
PLEASE DESCRIBE WHAT RAOB IS DOIHO TO COHPROHT THE VERY 
REAL X8S9B OP THE LACE OP STDDERT tfOTIVATIOH? 
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n National Assessment Governing Board 



Th© Honorable l>«l« E. Kildoe 
Chairtaan 

Subconmittoo on £Ie»Bentary, secondary and 

Vocational Education 
Comnitte© on Education and I^bor 
U.S. House of Representatives 
%faS!lin9ton« C.C. 20&lb 

Dear Mr. Chairaan: 

Thank you very murh tor the opportunity to testify betore your 
subcomroittoe on March n. Enclosed are my rc»pons«*r. to the? 
additional questions you forwarded to nte. 

Plea&e feel free to call upon mo, other ©envhers of the Board, or 
NACB staff if there ist. any way we can be of assistance. 




March 22, 1991 




herbert J. KaiWrq 
Board Member 



Enclosures 



I i<it; <;str.^ \ Sib 
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RESPONSES TO 
ADDITIONAL QUESTIONS FOR DR. WALBERG 
FROM CHAIRMAN KILDEE 
SUBCOTMITTEE ON ELEMENTARY, SECONDARY AND VOCATIONAL EDUCATION 



1. In your opinion, is the us© of NAEP as a "thcrmorocter" to 
measure the status of U*S. education compatible with an expanded 
uce of NAEP as a hiqh ctaKes national examination comparing 
performance o£ individual schools or individual students? 

Answer: 

The National Assessment Governing Board believes that NAEP is our 
best iseasurc of the status of U*S. education. The Board believes 
that the action by the Congress to authorize trial state 
assessments in 1990 and 1992 prudently expands NAEP*s ability to 
monitor the progress of U.S. education without "raising the 
stakes.** 

This expansion will help monitor state performance. It permits 
comparison among states^ among regions and to the nation of overall 
student results and student performance on various subscales; 
establishes base*line data for states to use in measuring progress 
over time? and provides useful background information on variables 
related to educational effectiveness Ce*g. teacher training and 
parent involvement). But this expansion has none of the 
characteristics of a state level program evaluation or 
accountability mechanism, as some have asserted. However, the 
assessment data can help states identify potential strengths and 
weaknesses in their programs that could then be used to guide their 
own program evaluation efforts. 

The Board has adopted a policy, described in detail in my 
testiaony^ recommending amendments to the NAEP legislation that 
would permit the use of NAEP and NAEP items by states and districts 
for reporting below the state levoX. such uses of NAEP would be 
at local option and cost, subject to uniform requirements for test 
administration, security and reporting, and would be separate from 
the federal NAEP program. The policy further stipulates that 
public reporting of individual student scores would be prohibited* 

In my opinion, such local uses of NAEP would not bo incompatible 
with its use as a monitor of national and state education progress. 



2. Does the National Assessment Governing Board seek more than the 
authority to do planning for state-level assessments beyond 1992? 
Given the absence of any evaluation of the state-level assessment 
effort to date, is there any reason for the Congress, at this time, 
to consider permanently authorizing state assessments? 
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Ansver: 

The Board's position is that, unless th© congressional ly required 
independent evaluation demonstrates that they cannot yield valid, 
reliable state representative data, state-level assessments should 
be made a regular part of KAEP, 

However, with respect to the 1994 and 1996 assessaents, there 
exists an issue of timing. The award for NAEP operations for 1994 
and 1996 assessments is scheduled for KY 1992, The current NAEP 
statute, which expires in Fy 1993, authorizes trial state 
assessments only in 1990 and 1992. Thus, for the 1994 and 1996 
assessments, there is a mismatch between the award cycle and the 
NAEP authorization cycle. 

Providing explicit authority now for planning for state-level 
assessments in 1994 and 1996 would allow for needed preparations 
which, if indicated by negative evaluations of the trial state 
assessment, could be cancelled. However, if evaluations indicate 
that state-level assessments ^^re successful but are not authorized 
before NAEP reauthorization in FY 1994, there may be insufficient 
time to continue them until the 1996 assessment at the earliest. 

Knowing that this question is a concern of the Congress, the 
National Academy of Education (MAE) plans to issue in March 1991 
an interim report on the 1990 state trial assessment and a final 
report in September 1991. These reports, as well as the June 6, 
1991 release of the 1990 mathematics assessment results, should be 
useful in informing the Congress' decision. 

Dr. Robert Linn, co-chair of the NAE panel that is conducting 
the independent evaluation of the NAEP trial state assessment, 
provided a preview of the NAE evaluation in testimony before your 
subcommittee on March 13, 1991. In discussing the evaluation, he 
said that "...1 can say that the panel has not identified major 
flaws in the sampling or administration of the Trial State 
Assessment that should preclude the release of results as scheduled 
in June. I can also report that the initial experience was 
sufficiently positive and without adverse effects on the national 
assessment that the panel will recommend continuation of an 
expanded trial in 1994." 

It appears that NAEP will be an important component for reporting 
on progress toward national educational goals. In addition, 40 
states have already indicated their intention to participate in the 
second trial state assessment in 1992, and we estimate that the 
total will be 4 3-45 states. This indicates a high degree of 
interest on the part of the states; we believe that this interest 
reflects confidence in the first trial state assessment and the 
plans for the second, and is likely to continue. 

Thus, timing of the new NAEP award, the likely positive 
recommendation of the NAE panel, the continued high level of 
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voluntary participation by the states, and the need for state 
representative data for reporting progress on national education 
goals, all seem to argue in favor of the Congress considering 
action to permit NAEP to collect and report state- representative 
data for the 1994 and 1996 assessntents. 



3. Please describe the process through which the National 
Assessment Governing Board set its Basic, Proficient and Advanced 
thresholds for performance on the sath HAEP assectsment . How did 
the Board measure the relative difficulty of individual questions 
on the NAEP assessment? How should these thresholds be interpreted 
and used by policymakers? 

Answer: 

The Board is in the process of writing a summary report on its 
process of setting achievement levels of basic, proficient, and 
advanced in mathematics on the 1990 national assessment* That 
report is still in the draft stage and a copy will be sent to you 
as soon as it is completed. The following is a brief synopsis of 
the process followed in setting the achievement levels* 

At its December 1989 meeting, the Board adopted a motion to proceed 
in principle with the setting of achievement levels for the 1990 
mathematics assessment. A public hearing was held in Washington 
in January of 1990 at which nearly 25 individuals and organizations 
presented oral or written testimony. Subsequent to this hearing, 
the Board adopted a plan to set the levels and proceeded to 
implement that plan over the course of the next several months. 
Please see Attachment 1, a copy of a memorandum from Roy Truby, 
NAGB Executive Director, addressed to Governor Roy Romor, dated 
December 19, 1990, which summarizes the process from May 11, 1990 
until December of 1990. 

Subsequent to the activities detailed in the memorandum to Governor 
Romer, the Board decided to collect additional data on the draft 
mathematics achievement levels. This decision was, in large part, 
a result of comments received during the Board's third public 
hearing on achievement levels held in January of 1991, The 
additional data will come from a replication/validation activity 
that the Board now has underway. The replication/validation 
activity will occur at four regional sites (Michigan, Florida, 
Connecticut, and California). We were advised that more classroom 
teachers should be involved and have acted accordingly. At each 
site approximately 60 classroom mathematics teachers and 
approximately 12 non-educators will participate. The 
replication/validation activity will result in ratings from almost 
300 individuals, about 240 of whom are classroom teachers* 

Following the conclusion of these four regional meetings in April « 
the draft descriptions for each level may be modified* The Board 
intends to take f^r^l action on setting the achievement levels at 
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its meeting in Washington on Nay lO-Ii, 1991. 

The Board used two seans to judge the relative difficulty of 
various individual questions on the NAEP assessxnent- (1) by asking 
IHinelb of expert judges to rate independently the difficulty of 
each question at each of the three grades tested by H/lEP, and (2) 
by having the expert panels examine data showing how many students 
at each grade successfully answered iche questions on the 
assi.asiaent . 

Achievement levels in mathematics wiii provide policymakers for the 
first time with an informed judgment about how much of the content 
of the mathematics assessment students at various levels of 
achievement and at various grade levels should know. Prior to this 
point, policymakers, as well as others, including parents, and the 
general public were only given information about what students did 
or did not know, without any benchmarks or reference points for 
judging the adequacy of this performance » 

In addition, having three levels {basic^ proficient and advanced) 
will permit policymakers to monitor the distribution of 
performance, not just average performance. This will assure 
attention to both high and low performing student populations and 
permit setting goals and monitoring progress within each 
achievement level at each grade. 



4. Please doscribe the consensu?^ process which NAG6 goes through 
to determine test specifications. 



The consensus process that the Board uses to determine the content 
and test specifications for e^ch assessment is comprehensive, 
broad, and inclusive. It begins with the Board issuing an RFP 
detailing the steps, processes, and procedures potential bidders 
must execute to assist the Board in accomplishing its statutory 
task. Should the subcommittee want, I would be pleased to send you 
copies of the RFP work statement and/or any of the documents 
produced as a result of the consensus process. 

Again, and very briefly, the successful contractor convenes both 
a steering committee and a planning committee of experts am' 
knowledgeable lay citizens to supervise and guide the consensus 
process. Routinely, public hearings are held, expert testimony Is 
collected, interested parties are consulted, and reviews of 
recommendations are conducted prior to the Board's final a^'tion to 
approve the content framework and the test specifications. 

The result is a very detailed and extensive description of what 
should be included in each assessment. This description is used 
to develop the actual items that will be used in each test. As an 
example, I am attaching a copy of the Mathematics Objectives for 



Answer: 
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thtt 1990 AflsofiSMnt (Attadment 2). 



5. Hhat research «videiic« is thera that torsal student aasesssents 
and their remits actually lead to conges in the behavior of 
atudenta and teachers? Are those changes productive or are they 
StiMtiMS counterproductive? 

Answer: 

There is a great deal of research evidem^e, as %fell as anecdotal 
evidence that, when properly used, foreal assMraents lead to 
changes in behavior of students and teachers* We know, for 
exaaple, ^en teachers are trained to use assesssents as diagnostic 
tools, that classroos instruction can become wore individualised 
and that individualized instructional strategies aiead at 
reinforcing student strengths and overcoming weaJcnessea frequently 
result in improved student performance « 

However, under the present requirements in P.L. 100-297^ which 
prohibit reporting national assessment data on the individual 
student, school, or district level, it is highly unlikely that any 
particnilar teacher or student will modify behavior based on KAEP 
results* 



6. Has KAGB or NAEP estisated what it would cost for a national 
test given to each child? 

Answer: 

NAGB has never maintained that HAEP should be administered to each 
child. Therefore, NAGB has not estimated the cost of converting 
HAEP froB an assessaent based on a representative sample of 
students in grades four, eight and twelve to a test of all students 
in those grades^ 



7. With the increased attention at the state and local level in so- 
called authentic assessment ^ that is tests of what students 
actual iy know and can do, to what extent has tiAEP incorporated any 
of these new aasesament techniques? How do minority and 
disadvantaged children perform on authentic assessments? 

Answer: 

The term authentic assessments* generally refers to testing 
methodologies in which the examinee produces responses, as 
contrasted with choosing a response in a multiple choice teat item. 
Both types of items, if properly designed and scored, can be useful 
in describing what students know and can do* Assessments that use 
both types of items increase the range of skills and knowledge that 
can be assessed* 
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NA£P has much experience authentic assessment. As early as 

the 1970*8, NAEP developed innovative approaches to assessing music 
and the visual arts, which included observations and evaluations 
of individual and group student perfonoance. NAEP writing 
assessments involve solely the production of student writing 
samplesr Approximately 40l of the 1992 reading assessment is 
devoted to student produced responses. The 1992 reading assessment 
also includes a study that involves direct student interviews and 
portfolios. The 1990 mathematics assessment included items to 
assess highiir order thinking skills, computer skills, use of 
measurement instruments, and calculator applicationsr Planning for 
the 1994 science atx^essment is now underway, and it is likely that 
it will include a balance of multiple choice and student 
performance items. 

We do not know conclusively how minority and disadvantaged students 
will perform on ••authentic" type NAEP assessment items. However, 
NAEP trend data for reading, mathematics and science from the mid- 
1970 » 9 through the mid-to-late 1980 's indicate a significant 
reduction in the gap between the performance of white and minority 
students which holds true across ages 9, 13 and 17. The minority 
gains at the higher ages, involving assessments of incre^^^ingly 
complex subject matter, could be a positive indicator for 
performance on "authentic" items, but this is not certain. NAGB 
will be following this issue closely. Clearly, state- 
representative data on this issue will provide state officials with 
information necessary to determine whether state policies and 
programs need evaluation or revision. 



8. Please describe the process NAGB has established which ensures 
that questions are bias free? 

Answer: 

The Board has developed a comprehensive policy that is used in the 
development and review of cognitive i* ms for each assessment. A 
copy of the policy is attached (Atta^ ,ment 3). 



9. How does lack of motivation affect the achievement level of 
students, particularly high school seniors? Please describe what 
NAGB is doing to confront the very real issue of the lack of 
student motivation? 

Answer: 

The Board is concerned that high school seniors selected to 
participate in NAEP may not have incentives to put forth their 
greatest effort. For example, these students know that their 
individual NAEP results will not be reported to their schools or 
their families and will have no effect on their grades or 
graduation prospects. Currently, there is no evidence that a 
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problem exists, but the Board has askod the itA£P Technical Review 
Panel (TRP) » established by the National Center for Education 
Statistics^ to conduct research on this issue. The TRP is 
developing recc^mendations for a study design that iK>uld be 
conducted as a part of the 1992 assessments. 
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National Assessment Governing Board 



XO Cov. Roy RoBar, chair&^n 

^^ational Education Coal» Ponel 

FROM : »oy Truby, Executive Director ^"^K/^"^ 
Hdtlonftl Asscsssent Covorning Board 

DATE : Decenber 19, 1990 

SUBJECT i Summary of Achieveioent Level Setting Process 



On May n, 1990 the National Assessment Governing BoAra (NAGB) 
unanimously approved a plan to cset the first standards for student 
achievement on the National Assessment of Educational Progress 
(NAEP) $ which tests a representative cross-^section of students in 
grades 4, 8/ arvd 12. The plan had been umSer discussion and review 
since the summer of 1989? it vas a modified version of a proposal 
presented to the Board in Decetabor 1989 vhich was the subject of 
a lengthy public hearing last January. 

Under the plan NACB vill define three levels of achievement 
--basic, proficient, and advancod--for each grade and subject 
tested by the National Assessment. It then vill report what 
proportion of students have reached each level both nationwide and 
in participating states. The system is in contrast to NAEP's past 
practice of simply describing how student© perform with no 
reference to standards of how well they ought to do. 

The Board was authorised to identify "appropriate achievement 
goals" on NAEP as part of the legislation that created NAGB in 
1980. long before any national education goals were formulated. 
In adopting its policy, however, the Board r;ept clearly in mind the 
need to help tracX progress toward Coal Three, set by President 
Bush and the nation's governors, that "by tho year 2000 American 
students will leave grades 4, 8, and li having demonstrated 
competency in challenging subject matter including English, 
mathematics, science, history, and geography." In fact, the 
phrase, "demonstrating competency over challenging subject 
matter, ' has been incorporated as the main defining laivguage of 
the Board's description of the proficient level for each grade. 

To lis the division of responsibilities between NAGB and the 
National Education Goals Panel seems clear. NACB, through its 
processes of wide consultation with experts and the public, can 
define in considerable detail the content of what different levels 
of achievement ought to be. The Goals Panel can then set numerical 
goals and targets for the percentage of students that should be at 
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#4ich l«v«l by thm y««r 2000« Alternatively* the ^ncl 

•igbt Mt targets and « ti»«t«bl« for the Inc^g^g^ in th* 
proportion of studonts ftt or abovo oach ochiovsMnt lovol« Thus, 
to90th«r, MXCB «nd tho Notional ediicotion Ooalo Panol oon sot goals 
ond targets to givo oubotontivo Maning to Goal Throo. 

NAOB iff sotting achiovojiant lovalo f irot on tha 1990 Rational 
As^eMoant of aatheBatics, fi^icXi i» ptroviding tha f irat atatO'-laval 
date ffchool parforvanco that la baaad on c^parahla, 

rapraaontativa sacploa of atudanta. Tha Board OKpacta to define 
achieveeent levela on the new NAEP testa of reading and writing for 
1992 end in acience, u«S. hiatorY# ond geography for 1994. 



KAGB understands veil that tha judgmental process used to 
define what achievei»ent ought to be should include a vide range of 
interests and perspectivea. Since this etandard^setting effort is 
perhaps the largest and Bost inpcrtant ever in A&erican aducation, 
the Board felt it must be open to public scrutiny and input. 

The following is a sumiBary of the process for setting math 
achieveaent levels s 

K ppoiBtoant of Advisory ^aMlr*Jttno_1990 

NACB appointed a panel of 63 judges. About 70 percent of the 
panel menbers were educators, representing subject area teachers, 
college Bath instructors, principals^ and state and district 
curriculum specialists* About }0 percent were non -educators-* 
employers, civic group representatives, and interested cititens. 
About 20 percent of the panel vera sinority group aeiabers. 
Attention was given also to gender and geograpliic&l representation. 
Panelists case free eehoole fro» New york to California^ froa the 
inner-^city schools of Detroit and Chicago, also free tha suburbs 
of tfinnetka, lUinois and scotch Plains^ Hew Jersey. They 
represented every part of the country and nearly ev^ry sub-group 
of the nation* a population. 

varmont Meeting— Awniat is and it, i^yg 

Standard setting is a judgmental process. This vesting In 
Essex Junction/ ven&ont, provided background and a fraraework for 
the 63 panel ses^rs to share their judgments. The meeting 
included the following: 

o Judges received training about the process. 

o Panelists u^et in four small heterogeneous groups at each 
grade-level 4, 8, end 12. The groups were given the actual 
test items fros the 1990 sath assessment. Each judge was 
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nnf< t« nalc* a lirat raui^ of rfttin9s, ir^icatinq what 
proportion of •tudontc At •ach •chi«vmitt l»vol ^Mht to 
«nsv«r Mch pArticular auction corrvctly. ThoM ratings vore 
d^ro^fttod cna AvorA^od to protfuc* a r«coaMndo4 AVcraae 
p«rcont corroct for •ich •chiovMont lov«l at oach qradt. 

o Th» 9toup» ver« qivn inforMtioA on hov studsntB ftctually 
P«rfor«od on MCh qMvtion during tho 1990 taotins. E*ch 
)ud90 tb«n did A gg«md rwnd of ratings. During this rmind 
» .ttXa or no diacuaaion took placa. Bavin? parformanca 
inforvatioft causad vary littlo ovarall chan^ in tha ratings. 

o Judges coaplatad a UUx^UtfiWia of ratings, Thia tl»a 

tha ^udgaa participated in a discussion about their first tw© 
rounds of ratings. 

o Tha resulta of tha third round of ratings vara ahar«d with 
tha total group of judges from all thraa gradea. 

yoit-Vtra^at HtaUag 

o Raviaions vara mada in sova proceduraa based uppn 
discussions with the technical advisory coaeittee on 
standard setting, ivo concerna vera given apecial 
attentions 

i^^^?*^*"^ •^f* jwdgea had a clSAr understanding of the 
Board's general definitions of basic, prof icient/ and 
advanced. 

(2) Enauring that judgaa baaed their ratinga on the 
difficulty of test ItaM and their i^F>ortance in shoving 
saatery rather than on vhether an ites or ite» fer»at vas 
'appreciate* for maep. 

o Analysee vere conductod of the ites ratinge to be uaed 
et next panel aeeting. 

rirst ■aabiaetoa M^tipq^^f^^f ^g aa^aa- \f ff 

o Thirty eight of the O judges reconvened in Kasbington. 

o entire group held a tvo hour diacuaaion to clarify 
the definitions of basic, proficient, and ^snced. 

o Judges CMplsted a touiXh rtaind of rating individual 
gueetims* 

? fS22* ^ individual grate-level and in groups that 
included panelists frott all throe grades to diacuss 
consistency end cdierence of raco»andad levels. 



in 



o Jud9«» »ftd« m jif^H round of ratln9s 9ivln9 th^ ov«r«ll 
p«rc«nt correct that ou9ht to t« r*qulrod to r«ach eacJ^ 
•chievcMnt l«v«l tor tholr 9T«d«. 

o J<id9«» coap^oted *n ovftluation form expressing their 
confidaRca l«v«2s in th«ir fimil ratings. 

paaond Was ttitMton tlaa— Koifoabst 11 sad IfgQ 

o A sssllsr group of 11 judgss wrots dsscrlptions of th# 
thres schiovcsent Isvsls for ssch grsds bssod on analyses of 
individual itsB ratings and avorags p^rcentm corroct dsrived 
fros judges' rating fornis at earlisr settings. Saspls itoiss 
ware salacted to illustrats each proposed level. 

o Final recommendations wsre cent to all panel ©embers for 
approval , 

HhS B Board Msstipg ia A t lantA—Kgys ^r 1» and 17. 

o The recoonendcd achievenont levels vers presented to NAGS 
at its sooting in Atlanta. The Board also heard comments fro» 
the project *s lead consultant. Professor Ronald K. Kambleton, 
of the University of Kassachusstts, and fros the lead 
eva)oator< Professor Daniel Stuff Isbeas, of Western Michigan 
University* The ovaluators reconnended moving forward to 
cospletion but cautioned HACB to proceed slovly enough to 
allov extensive public input. 

o first public hearing was held on Hovember 26. 



o Second public hearing will be held in Washington on January 

a. im. 

9 In sid-January an interis report by tea]» of independent 
evaluators will be sade available to NAC6 and to the public. 
Evaluation group is conprised of Stuff lebeas (Western 
Michigan), Richard Jaeger (University of Korth Carolina), and 
Michael Scriven (Stanford). 

o On January 23, 1991 the NAGB Executive Cocusittee and 
Achievttsent Levels Cocuslttee will review public conment 
— fros two hearings and written co&sent5-*and the interim 
evaluation report. 

o In late January or early February of 1991 the NAGB expects 
to BaKe a final decision regarding the satheeatics achievctaent 
levels. Xf the levels are approved, the Board plans to issue 
a report that displays data fros the 1990 sath assesssent in 
terss of these achievement levels. The report would be in its 
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ovn booklet. ftopArst* fro9 National Cantor for Education 
jtatiatiea (HCSS) raperta «i fiati^tftl fti^ atata NABP raaulta. 

o 9y r»bniary IS. 1991 aacb participating atata say tfaciaa 
vhathar to hava ita ovn 1999 NABP aigtith ^ratfa Mthaaatica 
data praparad for uaa by Xhm Govaming Board in tha JSfMB 
raport on a^iavawnt lavala. l^ftar inforvation ia praparad, 
atata auparintandanta vill tMva a cnanca to approva actual 
data for ralaaaa. 

o Tha national and atata data baaad on achlavasant lavala 
could poaaibly ba ralaaaad aa aarly aa Juna 1991 or at the 
lataat on Saptaabar 24, 1991 « tha aacond annivaraary of tha 
Charlottaavill* Education Suaait. 
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DAnif^i rt Koreir 
Senior JJiJci.i] ScUnti^t 

D*'Jir Dr Kor*'tr 

ffjTsh I?, iv^I hei^r iKf. Kit ittt' r>\jt>c*.auBt t cn tN-tw-nrirv. 5** ontJ*Trv . 
Ami VtK at ion.%1 Eitui aTitii^ TTiv f. una 1 T t f f* rptPlvf'd a vealih of 
v««lu4Hl«« itil cra^f ion whi> h will p^rrArW jiK&isf our pffori^ tu rpvli>w 

^lt*jt.t' 2 it'.Jl {!.(* jJ*}itlunttt iju<>!rt i ^'iis whi\ti I i itJ ; t. ji t I'vi I 
li}Trf«--wt iu xiii' i%\ytp I w.'wti! ,atc /ijvtr .»• i *,t ar.. *- in rctuTi.it.i 

pftrtt**'J «i.jit ltlv 

Af,4i«. th.tnk y<>u fui v.tuj vjjtijhlt* If"-IiJ^f^nv 4nd vwur a«.Mvt.iiur in 
rx(«-dlt if-.f, th«' rt-:.»rtt >*Mir rti>%w«rv to i h** o.Mint'<,.il '^'l^^t^t.fls 

S Ifu rrcl V . 
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QUESTIONS FOR DR. KORSTZ 

QgSSTIOtIS 

1) Xli yO0H OFXHION* XS 7HS USB OP NA6P AS k *TUfiRMOM&TSR" 
TO KBASURS THB STATUS OF U.S. EDUCATION COMPATIBLE WITH 
AH BXPANDBD USB OP NABP AS A KXCH STAKES KATIONAL 
8XAMINATXON COHPARIMG PBRPORMANCB OP INDIVIDUAL SCHOOLS 
OR INDIVIDUAL STUDBNTS? 

2) IN rOUR TBSTIHONY, XOW STATED THAT USING NABP FOR 
ACCOUNTABILITY MIGHT UNDSRMINB IT, BECAUSE PEOPLE WOULD 
STRIVE TO RAISE SCORES. BUT ISN'T THAT EXACTLY WHAT WE 
SHOULD WANTx FOR STUDBNTS TO STRIVE TO ACHIEVE MORE? 



3) IN YOUR TESTIMONY. YOU STATED THAT NABP CANNOT PROVIDE 
RELIABLE INFORMATION ABOUT THE BPPBCTI VENBSS OP STATE 
OR LOCAL EDUCATIONAL PROGRAMS. MANY PEOPLE ARGUE 
OTHERWISE. PLEASE EXPLAIN IN MORE DETAIL WHY YOU 
BELIEVE THAT NAEP CANNOT SERVE THIS FUNCTION. 

4> WHAT ARE THE MOST ACCURATE COST ESTIMATES OF A 
COMPREHENSIVE NATIONAL TEST? 
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March 20, 1991 

Hunorabic Dale C K}lde« 
(*bairman 

SuhcommilU'i* on Elementary . StH*tmdary. and \*i>c:itii>nal Kducntion 
Committee on Education and Labor 
U S House of Representatives 
Washington, D. C. 20515 

Dear Mr Chairman: 

In response to your request. I am wntin>» U) answer the 
supplementary questions you M»nt me to follow up on the March 13. 1991 
heanng on national assessment i&sue&. I am plesM'd to have had the 
chance to be of assistance to you and the Suhrt^mmitti^ as you work on 
these issues 

Question 

in your opinion, is the use of NAKP as a thermometer" io 
measure the status of U S education cfimpatible with an 
expanded use of NAEP as a hjgh-stakes nationnl examination 
companng the performanct* of individual st*ht»oIs or mdiWdual 
iitudentfi'^ 

Answer 

No; these uses are incompatible Kvidfrici* continues to accumulate 
showing that a single test cannot simuHanefiusly 8<»rv'e both as a high- 
stakes tool of itccountability and a truhtw<^rthy tndicatar"a 'thermometer"*- 
(or gauging what our students know 

A primary reason why the«ie two functions are incompatible is that 
when tests are used for accountabthty, educators begin to focus attention on 
the specific content of the teat The content of any single test, however, is 
just a small sample of what we want stwdi'TJts to know We infer from 
students* performance on that small sample what they know about the 
much larger "domains ' of achievement that are really uf concern to the 
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public and to policymaKerB We are nv\ concerned with the performance of 
high school seniors on the rouflhly 240 specific items that constitute the 
NAEP science Xesi; we are concerned with what that impiies about students' 
mastery of biology, chemistry, physics, and so ou When educators begin to 
focus on the specific content of the test* that generahzation from the small 
sample of test items to the big domain of achievement is undermined. 
Scores become inflated 

In my view, extending the NAEP to provide comparisons of individual 
schools c^uld undermine the value of NA£F as a thermometer even if no 
explicit sanctions were attached to it. BecauM? of the current widespread 
use of tests as mdicators of school quality, we have seen many instances 
over the past decade where simply publicizing scores was sufficient to 
coav^rt a test into a "high stakes" examination Providing comparisons of 
individual students would constitute even more of a risk, but it would also 
require a wholesale alteration of the test, because the NA£P*s current 
design does not provide reliable estimates for individual students 

Question 

In your testimony, you stated that using NAKP for 
accountability might undermine it. because people would stnve 
to raise scores. But isn't that exacily whyt we should want for 
Students to stnve to achieve moi-e*^ 

Answer 

1 certainly agree that we need to Hnd ways to encourage studenta to 
strive to achieve more The question, however, is whether using NAEP for 
accountability ts a sensible way to accomplish that goaf 

One reason not to use NAEP m thi*i manner is the problem noted m 
the prcvioxis answer: doing so will undermine the value of the NAEP as a 
thermometer of the achievement of our nation s youth That would be a 
high price to f»y» because we currently have no reasonable substitute for 
the NAEP as a thermometer. We have no other assessment that testa large. 
natioBttily repmentative samples of students at frequent intervals in many 
subjects. If we wait to know whether or not the nation's educational 
achievement is improving^ we need to pres;er\'e the NAEP as a thermometer. 

A second reason not to use the NAEP for accountability is that teat- 
baaed accountability has not shown ttself to be an optimal way of 
encouraging studenta and teachers to learn more Rather, it encourages 
them to raise scores, which sometimes does and sometimes does not entail 
commensurate increases in real achievement Moreover, it sometimes has 
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pernicious side-effects, sucb as narrowing of iii^truciion and excessive drill 
on material ck^ly related to the test. 

Question 

In your testimony, you stated that NAEP cannot provide 
reliable information about the eflectiwness of stale or local 
educational systems Many people argue otherwise Please 
explain in more detail why you believe that NAEP cannot serve 
this function. 

Answer 

The specific reasons are numerous, but the gt»neral reason is this: 
many factors will cause states to differ m terms of lest scores, and the 
NAEP is simply DOt designed to sort them out 

This becomes clearer if one starts with an example outside of 
education Recently, the press reported that the mortality rate for adult 
males is higher m Harlem than in Bangladesh Would one infer from that 
finding that hosf^tal care m Harlem is mfenor to thai in Bangladesh, or 
that one is better off to become seriously ill in Bangladesh ih^ in Harlem? 
Probably not. There are a great many factors that enter into adult male 
mortality rates* including, to name only a few, drug and alcohol use. motor 
vehicle use» access to handguns, access to pnmary health care, and the 
quality of hospital care. The difference in mortahty rates has many 
plausible explanations other than the quality of hospital care 

Similarly, many diverse factors enter into differences m test scores, 
including, to name a few, parental education and aspi rations, poverty, 
nutritional and health status, family composition, community values, and 
educational quahty The NAEP is simply not designed to disentangle tht^se 
influences. It includes only limited and weak background inforroatum -for 
example, no mformatioD at all on individual uicoroe or ptn'erij', only weak 
information about school-level po\Trty. weak information about parental 
education, and extremely limited information about community 
charactenstics. One way to offset partially same of these weaknesses would 
be to uae longitudinal data that permit one to track students* growth over 
time, but NAEP lacks that as well Finally, NAEP collects only limited 
information about educational policy and practices As a result, even when 
differences among stales do reflect educational factors, we will ofVn not 
have any clear idea U'htch educational f;ictc>rs are to bhimc 
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Ignoring these limitaticms and treating NAEP results-for example, 
the results of the Trial State \»9essioents-'a« indications of diflerences in 
educational effectiveness runs the nsk of emulating ineCfective policies and 
eliminating elective ones. 

What are the most accurate cost estimates of a comprehensive 
national test? 

Answer 

This question has no simple answer; it depends on the test's 
characteristics. Performance assessment, for example, will cost more than 
conventional multif^e-dKiice tests because of the added costs of task 
development and scoring. Additional test security also adds coats, by 
increa^g logiatical ot»ta and often by requiring the development of 
additional test materials. Truly standardising administration, to avoid bias 
from incorrect test administration, fbrtiier increases costa; that is one factor 
that makes the current national NAEP more expensive than ofl'-the-shelf 
commercial teats administered by tethers. 

Many of the impc^tant coats of an additional comprehensive national 
tesf however, are non-finandal> As it ia. our nation has an unusually short 
scho^ year and fiDs an apprsciable part of it with test-related activities. 
The time required for testing itself is oiten a small part of the burden, 
because schools ollen devote substantial time to test prefmration. As we 
ocmtinue to add additionfti layers of testing, we need to ask about the time 
and local resoorees th^ require, what activiti^ are being su|i^aDted, and 
v^tber srhoehng will reaUy be improved as a result. 



RespectAilly, 

^ . 4 

Daniel Koretx 
Senior Social Scientist 
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^. COMMITTEE ON EDUCATION AND LABOR 



Robort L Linn. Ph D 
Profe&sor 

Vnlveisity of Co I oi Ado 
C«Qpu9 Box 2^9 
ftouldcr. CO 80J09 

Dear Dr tlnn 

TKank you ve?y auvh for the fitc^'llint tcMl»«tiy you presented at tK*« 
March 13. hrarlng of thp Subcoflaslttpp on EWmentary. Secondary. 

»ml V<M:»Monil Education The Subc<WBsttC«'e received a *«.aUh uf 
valuable Intoiaatiun whi.h wiU greatly assist <n*r effort* to r-vtrw 
the National ftssrs^oent of Education Progress 

Enclosed plt»as« fln4 thp diltSitlDnal questions which I indicatid 1 
vould be «fcklf»R you ro respond lo lor the record Dup to thi- ti^at 
Interest in the Issue. I vould «ppr«<-tate your assistance In retuminj; 
Che ansvered questions bv HaxcU 77. &« the hearing record can he 

prlnc<Ki quickly 

Again, thank you for vour valuable tcfirlmony and your Assistance Iti 
expodttinft the return of vout answers to the additional quP5tlon^ 

Sincerely. 



Daf© t. Knd*«0 
Chalrt&an 

dt 

enclosure 
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aUESTlOHS post DR* LXItN 



QOSSTIOHS 

1) CAR TOU DBSCRIBS SOMS OF TBB RBACTXORS Y08 HAVK 
RSCSXVBD PROM S7QDtST8« SCHOOL TBACBBR8, 

ADKSNXSTRATORS, AND POLICYMABBRS DORXNG THB TRXAL STATB 
ASSBSSMBIiTS? 

2) DO WB RON THB RISK OP OVBRBORDBRXKO OUR STUDBNTfi AND 
TBACHBRS WXTR TBSTXBQ7 BO^ MUCH TXMB XS ALRBADY 
CONSDN8D BY TESTING XN THB AVBRAOB STOOBliT'S ACAOBMIC 
YBAR? ROW MtJCH HORB TIMB MIGHT BB TABBB 0? WITH 
TBSTXMG IP MB MOVBD TO A STSTBK OP RATXOMAL A58B88MBNT 
OP ALL STPDBMTS? 

3) XM youR OPIMXON, 18 THB USB OP NABP AS A "TRBRMOMBTBR* 
TO NBA80RB TBB STATUS OP 0.8. BOUCATIOM COMPATXBLS WITH 
AH BXPAHDBD 08& OP MAEP AS A HXOH 8TARBS BATIONAL 
BXANXKATXON COMP%RXNO PBRPORMAHCB OP INDIVX00AL SCHOOLS 
OR INDIVIDUAL ST0DBNTS7 

4> HOtf DOES LACK OP MOTIVATION AFFECT THB ACHIBVBMBNT 

LBV8L OP STUDBNTS, PARTICULARLY HIGH SCHOOL 8BNIOR87 
CAN YOU DBSCRIBS WHAT HASP IS DOING TO CONFRONT THIS 
IMPORTANT ISSUE IN TUB STATB TRIAL ASSBSSMBNTS7 
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BAsponfitt to giiMtiras po9*4 by 
8ui>eosnittee oa Bl«&ettt«rf , Sooond«xy« 
CraalttM on Mvoatim fts<l Xi^bor 

U»5* SO«M of B^Mftttt«ti70S 

Robort LiBA 
Profossor, trnivorsity of coiortdo mt Bouldar 
ud Co-Dir«otor# center for Sostmre^ on BvaXuatioc^ 
8t«adarA9f studant Tostiag 

lUTon ai# 1991 

1) CAN YOU DESCRIBE SOME OF THE RE^CTIOHS YOU HAVE RECEIVED 
FROM STUDSKTS, SCHOOL TEACHERS, ADHINXSTRATOHS^ AND 
POLICYNAKHRS DUEIKG TH£ THXAX^ STATE ASSEdSKENT? 

Only prolioinary information is availaJbl« at this timo 
basod on observations of soso administrations, analyses of 
questionnaire responses from local administrators of the 
asseaamentx and a survey of state testing directors. 
Additional survey data will bo collected from policymakers 
following release of the results in June and a more complete 
report will be included in the October, 1991 report of the 
National Academy of Education Panel evaluating the Trial 
State Assessment. The ways in which the data are received 
and used by policymakers, the press, and the public will 
provide one of the acid tests of the usefulness of the Trial 
State Assessment. Based on preliminary results, however, it 
can be said th9t. tuost reports indicated that the 
administrate c.n of Trial State Assessment went well. Moat 
state tesnxng directors indicated that they thought that th« 
Trial State Assessment results would be of value to their 
states.. 

School and student participation rates provide an 
imtirect indication of reactions to the Trial state 
Assessment. Both school and student participation rates were 
good. Approximately 94% of t&e sampled schools agreed to 
participate and substitute schools vera successfully obtained 
for some of the refusals. Approximately 94% of the students 
randomly selected and askad to participate agreed to take 
part in the assessment* As la now done with the national 
NAEP results and other surveys, statistical adjustments will 
be made for non^response. By way of oomparisw, it might be 
noted that the corresponding grade 8 participation rates for 
the 19BB natimal reading assessment vere 86*6% for schools 
(prior to replac63oant) and 87.8% for students {based on the 
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1990 HASP npert "Learning to RMd in Our Natim's Schools**^ 
Tabla A.X). 

2) DO I9B ROH THE RISK OF OVEIUWRDSiriKO OCR S T UDKWy S XHD 

TSACBBRS WITH TESTIS;? WM mCR TINE IS ALREADY CONSUMED 
BY T£STXK<; IH THE AVBSAGS STOUEHT'S kCADWlC YEAR? HD9 
KUCH MORE TIHB XIORT BE TAKEN UP WITH TBSTIHO IF WE KOVED 
TO A SYSTEM OF SATIOKAL ASSESSMENT OF ALL STUDEIH^? 

Yes, ve run tlie risk cf werburdenii^ mir students and 
teachers with testing! students in sany eohool districts are 
required to take a standardised achievement test battery aach 
year. Depending on how comprehensive the test battery is, 
anyvhere fros 2 to 5 hmira of actual testing time say be 
required. The Hatienal Cmaission on Testing and Public 
Policy recently estioatcd that *eaoh year elementary and 
secondary students teJco 127 sillim separate tests as part of 
standardised test batteries sandated by atates and districts. 
{A test battery may inolude separate tests that masure such 
things as reading, oathematies, seiei^e, and study skills). 
At some grade levels a student may take as many as seven to 
12 such tests a year." In addition, students may be required 
to take district or stats tests as part of ninimun competency 
t fisting requirements and state or district assessments* 

The time spent in taking the actual tests, hoi^ver. Is 
but the tip of the iceberg vhen it moes to time devoted to 
externally mandated tests. Research conducted by my 
colleague, Lorrie Shepard, throu^ the Canter for Research on 
Evaluation, Standards, and Student Tasting has shown that 
Tiany elementary school taachers devote the equivalent of 
several weeks of class time teaching students test-taxing 
strategies. In addition, in situations i^ere accountability 
tests have a great deal of iaportance, teachers may teach 
content throughout the school year using worksheets that 
closely reamble stai^rdised test formats. 

Hov such more time might be taken up with testing if ve 
moved to a National Test of all students would depend on the 
nature of that program and whether it was an addition to, 
rather than a replacement for, sme existing testing. In its 
present form, MAEP requiree only about one hour per student 
assessed. Because different samples of students are 
administered different sables of assesssmit exercises, this 
is adequate time for the purpose of monitoring achievement 
trends for the nation or a state. Substantially sore time, 
say 3 or 4 hours, per student would be needed, however, in 
order to obtain reliable scores for individuals* If this 
were not only an add-on to current testing, but another test 
for which teachers felt co^elled to spend time providing 
students with special test preparation, the expansion of 
^tional Assessment to all students would add significantly 
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to an aln&dy excossive amunt ot instjnsctional tino davot^ 
to eternally mandated testing and preparation for thoaa 
taats. 

3) XJI YOTR OPINlOHi IS THS USS Of NAEP AS A »THERKOMETER* TO 
MEASUKE THB STATUS OF U.S. EDOCATIOJf COKPATIBLB WITH AN 
EXPANDED OF NABP AS A HIGH STARES NATXOHAL 

EXAMINATION CWPAftING PERPURHANCE OP INDTVIOaAL SCHOOLS 
OH IltDXVIDUAl' STUDENTS? 

Vol There is substantial experience with testing 
prograae that have been designed to serve multiple purpoeea. 
That experience indicates that a single testing prograo 
cannot sinultaneously serve well the distinct purposes of 
monitoring trends in achiaveaant, school building 
accountability, and the certification of individual student 
achievement. The accountability or certification fwctions 
result in a narrow focusing of attention on the test. As a 
consequence, generalizations about achievement defined mora 
broadly than just the items on a specific test are 
undermined. That is, the validity of conclusions about 
educational progress that can be drawn from an independent 
indicator system such as NAEP would be threatened by its 
expanded role as a high-ostakes national examination system. 

In addition to a narrow focus on the content of the 
testr high-staices testing programs are also subject to other 
pressures that limit their value as an independent monitor. 
As the staJtes go up, there are increasing pressures to narrow 
the content of the test. Law suits associated with high- 
stakes testing programs have a chilling effect that reduces 
tests to a lowest common denominator* Specifically, thoy 
must be limited to vrtiat certainly has been taught rather than 
to what ideally should be taught. If this were to happen to 
HHEP, we would no longer be able to monitor progress in 
important highor-order thinking skills and the ability of 
students to solve novel problems. Vet it is precisely such 
accomplishments that are vital to the future of the nation. 

4> HOW DOES lACK OF KOTIVATION AFFECT THE ACailEVEMEKT LEVEI 
or STUDENTS, PARTICULARLy HIGH SCHOOL SENIORS? CAN YOU 
DESCRIBE WHAT NAEP IS DOING TO CONFRONT THIS IJfPORTANT 
ISSUE IN THE STATE TRIAL ASSESSMENTS? 

Assessments of student achievment depend on the 
cooperation of students^ not only in agreeing to participate, 
but in the effort that they devote to the assessment. 
Questions have been raised about the motivation of students 
to perform their beet on an aMeaament that does not have 
consequenoee for the individual participants. NCES has 
created a Technical Review Panel through a contract with the 
Center for Research on Evaluation^ Standards, and Student 
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TMting awi tOi© lUMD Corporation to cxmduct validity studiM 
for N&EP* It is imtieipatad Uut studies of stiidmt 
motivation vill ba ona of tha priority aroaa for 
invaatigation. That i#ork is yat to ba dona, hovavcr. 

Student participation ratas by grada provide meme 
imiirect inforsation tbat la ralavant to this iaaue of liXoly 
diffarantial sotivation for hi0i school seniors in conparison 
to students at grades 4 or 8. In the 1988 reading 
assesament, for exasipla, the participation rates at grades 4, 
a, and 12 i#era 52.8%, 87.8%, awl 78. s%, respectivoly (ba^ed 
on the 1990 NA£P report 'Learning to Bead in Our Nation's 
Scl)ools<*, Table A«l)* Clearly , high school seniors are much 
more lilcaly to refuse to participate in HASP than are their 
counterparts at earlier grades. The lover level of 
iBotivation of seniors to participate in the aseasssent sight 
also be reflected in the effort of so&e of the students vho 
do participate. This difference in student villingness to 
participate is not surprising to anyone familiar with high 
achool seniors. Nor would it be surprising to find that the 
fraction of students who participate in the aeseesmant in a 
half«4iaarted sanner inoraaaed tram 4th to sth to 12th grades. 

Tn any event, there are sufficient differences bet%men 
high school seniors and studenta in the 4th or Sth grades 
that it would be unwise to saha generalizations about the 
likely sttccesa of a Trial State Asseesoent for 12th grade 
students froD results obtained for 8th grade students in 1990 
or 4th and ath grade etudenta in 1992 « lltiua, if Congress is 
interested in expanding state-levei use of NA£P to include 
the 12th grade, X would encwrage consideration of starting 
with a trial in 1994 along the lixiea of the trials authorized 
at the earlier grades for 1990 and 1992. 

Respectively submitted » 
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JOHN F. AKERS 
STATEnENT TO TRB BOUSB S>VCATIOK MID LABOH 
SUBO^QflTTES BLBMENTASy, SBOSiinARY AMD VOCATIQKAL SmJCATIQN 

09 TSS 

NATIONAL ASSBSSKBNT OP ETOCATIOK WBOSBSSS 



Chairman Kildee, my name is John Akcrs, chairman and chief 
executive officer of International Business Machines Corporation. 
I am submitting testimony as chairman of the Education TasH Force 
of The Business Rcundtable CBRT)* The BRT is an organization 
comprised of 212 chief executive officers of many of America's 
largest corporations. The BRT CEOs have committed their personal 
time and company resources to a decade- long effort to help 
improve the quality of the nation's R*12 educatior« system. 

The BRT has identified strategies we believe are essential 
to achieving the national education goals. I am submitting these 
strategies, 'The Essential Ccanponents of a Successful Education 
System^" for the record. We are working in all 50 states and the 
District of Columbia to address the essential components through 
legislation and regulation « 

Assessment is a key point in our agenda. To provide 
assistance, we need detailed information about what students know 
and are able to do. And, to hold educators and policy makers 
accountable for student performance, we must have data to measure 
it at the building, district, state, and national levels. Thus, 
we applaud the Senate's interest in assessment and recommend that 
you allow the National Assessment of Education Progress (NAEP) to 
expand to measure student performance at the state, district, and 
building levels* 

National testing has become a subject of significant debate. 
A consensus is emerging in favor of a national assessment syst&n 
that would revolve around a common set of standards for student 
achievement. Staces. districts, and other entities would create 
exams, either jointly or independently, to measure whether 
students are meeting the common standards. Until such a system 
is developed, we have to rely on existing instruments* 

NAEP provides a clear snapshot of what Americans students 
know and are able to do* Educators and policy makers applaud its 
focus on writing and problem-solving skills. 

NAEP is not perfect, but it is currently the best test of 
its kind* The National Education Goals Panel is considering 
using it as an interim measure. At their Winter Meeting in 
February, the National Governors' Association passed a resolution 
calling for the expansion of NAEP. 
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congress should remove some of the restrictions on NAEP. 
NAEP currently publishes its findings about national student 
performance In the aggregate and breaks down student achievement 
by sex, race, region, and other categories. With the exception 
of the 1990 and 1992 pilot exams, federal law expressly prohibits 
MAEP fr<Mn reporting student achievement by state. 

We urge you to extend the state-by-state assessment to 
include all appropriate subject areas at all grade levels. We 
also support lifting the prohibition against using NAEP to 
measure district and building perforroance, so long as it can be 
done within the context of the present n»trix sampling approach. 
The expansion of HAEP should be contingent upon a positive 
evaluation of the 1990 pilot test. Districts and buildings that 
participate in NAEP should be required to use the data to improve 
the performance of the most needy students. We do not support 
any use of NAEP that would allow for the comparison or evaluation 
of individual students. 

Quality assessment instruments alone will not improve 
student achievement. But without stronger and richer assessment 
strategies, America's education system will not reach its 
potential - 

We appreciate this opportunity to submit testimony and look 
forward to working with the Congress on this important issue. 

March 14, 1991 
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National Center for Fair & Open testing 



March 18, 1991 

Rep Dale Kildec, Chaif 

Subc^mjttee on Elemeniary ami Sectmdary Hducatfon 
320 Cannon House Offm Buildm^ 
Wjshmgfon, DC 205 1 5 

Deaj Rep Kildce 

rtiatik y«Hj for having invtted me to lestity on ibe issue ot a ruiwrnn^ lest Of exammatum 
system. 

In addiiion io ihe testiinony ! h-i\T vnhmmed on that i^ue. I request ihai the enclosed Open 
l.etirr to Congress. Bu^h Aiimifustranoft. the G>vcrnor> on NACia and NAFJ' F.^panMiin he 
tntmduced mio the reci>rd of the bejnng !be Natj^wial Assessment of F.ducationai Progress 
ot March 13. IWl 

The siaiemcni cndi>rsrd by a hmg h^ of education and civil nghts organizations 4nd 
tcadmg jfidividujls !e rcsp^mds w\ ihe cxpan^uv* profx^saK sel fon^ani by ihe Nai«>n3l 
Asveswnem Gt>vrTnmg Board i« December I W that were presenied by Bi\ard member 
Herben Walherg to the heanng tm N'AI'P 

Ihjnk >(Hi tor keeping t*)e retufd t'fvn jud intJudmg thjs \taiemeni 



YoUES. 




Monty \ei1l. Fd D 
A\S4Kiate Diretu^f 
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FairTest 



hiattonal Center for Fair & Open Testing 



OPCN LrTTTR TO CONGRESS. Bl'SH ADMINISTRATION. T^iF GOVI.RNORS 
ON NAGB AND NAEP EXPANSION 



Over ihc pa si vcvcral mtmths. the National A^sessmet^i Governing Ri^srd iNAGB> ha^ 
taken ^rvcfdl atJum^ uh;i.h, i:onM£icrcd ingcthcr. raise venous concern over the funiie 
directum ot the NatKwaJ Asseiismcm of EducauonaJ Prngre?;^ tNAFP) A^ a group t»f 
ediKMtion 3nd civtl n^h{> t>r^3n)7dii4.>ns di'uvc m school reform issues, we arr addrc^^ing i>ur 
iiintcm* \o Cun grcsv. the Administratian and ihe NjuoiuJ Oovemori Association so thai 2II 
rrNf«mMb)c p,trtje^ undcr>!and thr naiwr of xha^ problems and carrfylly mor.iior 
d^vcll^f>mcn{^ m NACiB jjid NAiJ^ It in ifnponant to note thai we art no! wntinj: 10 opfnisr 
the nation jt ass^NNmcrt. bus 10 help ensure thji it pl:j>^ a tonMructise. mrt harmful, roir m 
rcfomuni; our namm N cducatu^nal systems 

Tbc 3in<ms of ihe Governing Ikiard. tai^en togrthrr. go far beytwid the Icvci of actjv>t\ 
4«!hon7€-d in the Natuma) A>\e\smem of Educational Progress Impajvemem Act adiipted i\ 
pan o! ?hc Mjukjn\ SMiford Eiememarv and S^rcomUry F.duvaium Amervdmems of I9KS 
Tlwt Awl \V\. ?{S> r*^'f. ^bich passed tollowmg lcng}h\ di\i:usMi>n. aulhxKiTcd voluniarN 
stMc !u sutf crwnpAnMmN <if XAflP asscvsment results on a mal ba\is. and mandated an 
fntiffcndff't yUhU of the vahditv and effects of the piUs pmpnims 

i^sN {hjn vrurs laier. pnitr to ciimplctum of the mal cntnpartMmv and the ^JJ4i.e^, 
\ AiiH t> pMp< tstni: a ms'^ir ex pans Km vt NAf'P (\rc NAOB's p.tper PositK'nN <m the i uti/re 
i»i NjK.m.ti \sNCNsn'.eni > The proposaE ^n,.^ude^ U VjH panicipation h> ihe stajo m 
■ r.ifc -'v ^i^te V 4'n\*Mr;s{ms. to ^ht pud lijr hv the fedcrjl ^.>\ernT.fm. tesjinj: jn^jl u*n-<pir"-ni: 
;«>v.it 'Pvts .ir:J c\cn •».^<>{>lv, uhtth currrntK pri>^:Hi;cd h> Uu. ajid ^» more frcv^ui"^: 
H'vi:n.* I .t%t initnth. \ \(ill adopftrd J pftK ess for scitini? jvhjfseineni ieseK tf-.jt >r»iJf":\ 
• n cr.ul,-N t.*.,r. v ,tit jnd tAcKc 4M..:h{ to *m \ tc>;s 'vfc \ \<»li pjpi-r Sctuni' 

• r.uh ol thc< ipji.jlivcs rj.sCN pfohienw thjt n-tj.iire sfrkOii> jtlrnt:,* ). \vr mv 
;mi . .r.\ i.^ri. -nrit .t's'-.n &\r . mftn.tu^'n ft vcttiri: j.. *,*r\t''nrnt is-vris .ir.i / 

l) I hi' proptisa! \ti f\pand VALI* was adnpfcd bifnrc itimpUtitm of thr 
f 'onjirvssiimaH.^'mandaittJ ^tudirt or ihi* pilftt sf;»te*by -state compariMms. 

IApjnsM>n i»! NAliP ujU mevHab!> atfect our r>3tK>n\ edurjtton i'onprsv ii^rre.TH 
phmnrd a i..ui;5o;is. step h\'\tep pTivess to gauge the vjiue and rtfct t* of Mate cornp.inMns 
hcN*fo m^ndjttnj; (htr»r ».i»nnnu,i*.Km or evpanvion This evdijjium st^'uld W vi'^iplcted 
f>r}*>fr ar:v tunher strps .ur taken to expand \A!1P 

2 J N ACiR i\ pru^RAittj; t xpansum of N Al-.l* hirftiit- !ht» fVitiona! dihati* on 
edutatKioul goals is rcs<»hed, 

S*> laf, the Hush Administration and Ihe Goveni<>rN hj\r aj^reed on hf<»atl njiuir,.*; 
>!o.\ls, h;it hj\e vrt tt>dcL;dc to implement them U^'italiv. ;hc AdrnjoistraUim. the 
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Goveraors and CoogreM ihouW all have rotes in ihi* dchaa &s wtiljj^^ 
mdicaton tt«d » measure progress towird the B«t tf inc^./^«°^ f*^ 
cUrtficatioffi, the process erf rnewhng benxwncs, by def^iit ihc p^^i ^ ^< 
%MNild iniiy be putting ite can befcoe tbe tttsnc 

Dtfemng acnoo NAEP cxpanskm imtil after the tnsl ^^'^'''kx^ ^\j^la>» 
legally trquircd studies are compteted wiU aJtow lime for !t» f»atTi^a£^^^^*^itti%^ 
cducanonal goah 10 nw^ TCSoSutkm. Only il»n can NAEP pteyj a^p^^^/^ V 
developing sfpwpnatt messiirement tools and prpccdurrs 

3) It is rwklcfs lo ccmridcr llfWng the b«n on dfetrict-^j^\< ^ «to\*»t^* 
school comparisons wiihout considering Uie consequences for«:A^i^„^»B<g 
Insfructioo. 

No ra« yet know* ibc effects • and side-cffecn - even of jwj^J^ ^j^. ^Vy***^;* 
Repeal of ihe ban on Tocal comparison* require* much more uift<?T|^jj|^^^fT»u^J^ . 
discussion Ii shoiOd not be consideftd unnl after the results of ^fil^^^ ^^/Vo'^hj^ 
mandafed studies Have been fully analyisd and publicly diKussc^ ^ 

4> NAIJB's Adiievement selling procfiss, when oijtm^^/^ /*tl ^V^^p^^^A^^ 
may crctife a ds facto nafiona) curriettlunt. 

The evidence is omwhcfming th« the more power aIlac^ll<^^.^ ^^^.iJ* ^ ^* 
the test will have over cumculum and in^uuctjon. A natwrna! tes lAy^ 
and local cixmpansons will certainly become 3 powerful. pcrti3psjv^^/^^f>,s «,»^»c 
cumculum 

The cdmauon goals enuociaied by ihe Admimsiranon an^s ^^^^ 
aiicmpi TO mandiic a national cumculum In fact, there i? wideii^^x<^^\^^'^ ^ 
cumculum and instruction should not be determined from Washi^t^/ ^j^^^ '^•(f\ 
communiticik need tlextbjhty m detennmmg ho* to anain the hnspyi^ fA 
expansion pn>povai% could prciiudc state and local jfutjatjvcs 

5} NAC;b*§ achievement level setting procedures for is^^*^^*^ jcx^^fe^^of^^^p 
for determtntng na^ionat achievement guuts^ 

Tlx pTtvcss i:ho«n by NAGB to set achievement leveh E^stj^tj^j^^ \t!,n^ 
ticm\ fmm c^iMmg NAEP estams that, tn the vtew of commttien* «^ |/^^,^**.*<.f*^/^^}f r 
an!4wcrcd correctly by vtadents who have aifjjned iJw Irvels of Ix.^^' '^/^V^^^ 
'advanced " Thi** is r«« an approprutc method for dctrrmmmg «t:,'^}^^v^iS'^ ^ 
achie%-emfnt leveH because it allows one le^t to define the cortic^^jf^^^j^^ tf W . 
should be able to do tn that area. Such decision* should be mat> rf\p^^% ^S»»^^^i*V^ > 
of any tc« After cumcular goals have beett decided at the van»ij.t* ^rf' V ^-f^^ 

appn>pnaie 'o the cumculum can be constructed and achtevemca t^^t'^^ 

MortL ' . as the lecently-released report of the NanonaiClT^J/^^j^^eo** ^^^fr?^ 
Public Pohcy expbms. {he pnKTdure of relymg on c^nminecs ti l«si^\ -k^Nj-i^^ttti^* k 
fiawcU even fof the purpose of setting cut <^f scorri on sn^is \f ^f^J^ ^r^^' 

technical procedure to be adc«|u3tc for shapmg a national cumcii^i^ 

6^ By %^Un^ achievement goals bawrd on what are pr^*^^ \f^*^) »*/lj^f<h<^t-f 
tests. NA{;B runs the riyk of dt-nmnK national cducalional u^ii^ , f V 
Ihrse narrow instruments. 
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In p(Hemi4liy shajwng cumvuiufn and m>mjiUon. NAFP reM% wilt affect tHHh content 
and mcihodi «f tc^hmg Multiple ch4)H.*e testing nrccssanly ftKu^es an factual recall »mJ 
Riniple compansons and obscrvatmns U dow nm tend itself to revealing s*hcEhcr stydcnb 
know how to do wmcihmg - to wnte a persuasive citsay. research an histonc&l event, o* grasp 
the meanmg oi a urieniiftc development. 

The nam?ttnc«a of these mstniments hki been rfcognjicd by the G4>vcm<n^ anH>ng 
many others, and has ted to widespread etTons to develop ^ implement other means oi 
asvessmrnt If multiple thmce lestmg cj>nnnur\ to prrJommate. NAI P wtU provide a 
commual obstacle to reaching and a*vci>smg the impivruni thmgs students need lo leam ho^ 
to do. It will help perpetuate a reduced dcfinmon oi the comcni Ui he Mudted and an cmirrlv 
tncoTTPCt v tew of how students learn 

7) NAC'iB propo^e^ fo vq%ih intxeavr the ammint of lis twling to Include **al least 
three suhjecf^f each year." 

The Current NAKP juihon^Jiitm c>iab)jvhc\ j rwtvj^ar teMih^ cyxie and a minimum 
frequency icx JeMing vannus subject*. Only math and reading are id be ic*:*d every rwo 
ve3r%: tMher iuhiectJi are scltcduted at four- or iiR year intervals. Though us future* paper 
deterred di>cuv\u>R of the "ex«Jit crmfiguratatTfi" of the new testing cycles. NAGB called ft*r 
tonng at least three subjects rath year.' at least sin tests every two years. NACB\ i^Uims 
Xhis avcelcrdiion rv nccesiary lo pmvidf timely and sidficient data" and to "irplace the 
I'd'jcanon Dep.irtmenf*! annual 'wall chart' *hich rchei on SAT .ind ACT sccwrs " 

Ai;3>n. major changes :n NAFP such as expanding the extent and fiei^uency of teMmp 
should not be undert.iken pnnr to completion and analysi!^ of the testing and the 
mandated >tudjev In fact. <;uch expansion t% not at ali necessan- Because educational 
NSMcmv 4ud jLhiCvemem cannui ihani.*e rapidJv. yearly ap};Tv*pat;.d djta will not provide 
meaning! ul mJormatuin about jmponain educaittwai vhanjiev Less frcv^uenl mfi»rm4iitm 
>ti<Mitd he quite >uiricient 

While vjnitjlly everyone, mtiudmg Sct-rrlarv Ca^a/os. ai;rcrs on the inadcijuav\ "J 
tfK iurrcnr vhans. ' ihc mere cxKiente t>l ihe vh.uis ts an nuut fKient jusnfuaimn iiU 
vastiv jovrravrnji a natmnai tevtinji pni^am. T*i he surf, annual one p4>int ^.hanpCN in .^vffji-r 
HA'l scores or t*%o tenth> ot a point whan|:cs t»n the ACI m the ^saii chans ' are meaninj.':i*ss 
lint vuhstiiuunj: mmuse changes m NAIJ* V4.txri*\ vvvwlJ not he impffuemcnt U *,i>ui*.4 
htmever. prodiKc public fruvtninrin jnd thcTcbv jec>parJi7C puhhc ^upp^n ti>r ediivatu>n.il 
retnrm Ntamtutnmi: NAl'.P'v current. ajth<^n;cd svhcduie v\u\ pnnuir av mui^h uset.ii 
mUtmiJtJon at lcs\ cost m dnl!ar\ ^nd. ultffnjieJv, m pithhi cnrtiihtlitv 

Hi NA<tB is tnovm^ loo stonty in rcvtsinR NAKP e\atm in rtl\ k*ss tm multipte. 
choice qurslkms and lo develnp othtr mean* of assessmeni which better mcastire ihe futt 
r»ftfte of knowledge^ And skills 

While NAGK cbim^ that alxmt ^(K? tit this year's NAFP math items u-err 4>pL-n 
ended. Paul LeMjhjeu. PjttshurgnX Dirrttor ot lesttni;. informed the N'atumal ANSC<.ia!Mn oi 
Tcvt Diiiectiws that lesv than 5*y u-ere rrally <»pen endni ijem% The rcit uere muitjple-^hOKC 
quTMjonv *with the ansvwer oplK^v deleted Like multiple cbiMce iiemn. >uih questions are mn 
vet> useful m measuring \tuttent abilities to u^e math to solve rra! world pmhiems 

Instead oi e ^parnling the use of outdated, multiple choice tots. NAKP should become 
a leader in the nafumal effort to develi>p improved foi^tis assessment that provide more 
inf<innatiivn ^nJ do not endanpcr hut rjihcr enoth the cumculum NAI-.P should v^ojk ^uh 
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ihe swcs, a numtxr «t v^hich Alrrsdy hsit perf<mn3nce bis^d assessment pfojccfii under 
df vetopincni, to produce and evaluate such asseismenfs. 

9) NAF.P €%pan.sfon nitt ahM)H) an e^er targer share of federal rc.warth and 
mformafiun duilars hut thp results may not be utirth th« mone^. 

The NALJ* Impn»vcmcnt Act authonzai $9^SUD.a» (m fisKrai yrax I9S9 for NAF.P 
f-'^' ! W NAEP irveived $17.CIX4.0()0 Even wilh this mocascd amouni, the EdtKa!j<m 
Dcpamneni iidcnrd the NAEP stoJy. a nsitumdl asscvimcm af aduh ilijieravy and 

H<irK t>fl ihf NjiKwa) l-du<^:!iion UmgifiidjnaJ Study. For F^* NAGB h4!i irqucsfed 

$lS.Sf>h,0(Xl. an incrravc of monr ihun over FV^I and nearly doubic the auitumranon 
for NACiB irccivc\ up lo 10% <»f NAIli* '^mds for admimstranvc ptirpt->vcs and 

iriKwicdly srcks uj m'cive up lo 15% I'sCimar' « ot ihc cost vf NAi'P if cuponded arr $HK) 
nidlion annualjv, a more ihjn five fold incrcai. over curreni expendiiures and an amounr !*o 
jnd pne half times ihc lundmg for the Naiion i Ccnwr for Fducatiimal Siau^KS (NCtS) 

Will \hc TrMi\t\ be w<>nh ihr additiona ♦nomry'' Yrarfy tesimg will ntn mcrrasc 
anyxwS kno\^Icdf:c of ihe cffccis of edutauo al re{twni cfToris Funhcr state and Jocai 
4.<»mpanst>ns may m tcil u^ m?>Tr ihan we alira*! • V-.^nv about how wcU the states and 
hx\4l)fie\ pcrf*>jTn tm i»uinujfdi/cd tests. Irt a pemx.' of conunuing fibcal restraint, money uvcd 
for mi'tfr r^irnM\e testing unild he btrticr used to impmve the of NAI:P assessments 

i^r for <»ther ncrdei! nrse.itih rattjcf than for redundant and poiemially dangerous mcrpasrs m 

|0t rdaficmship amun}{ NAEP, NA<;B and M;F.S must be riarifit-d. 

The vunrnt dehaics over the future erf NAFP have raised queslKms about thr 
apprt^pnaiefTess i>f jn independent body ^^tcl.tJng the power that NAOB could assen over tmr 
naiutnS cduv^t»on A key issue whether vuch a btidy is adeqiiaiely atvouniablc 
<*im|2te!>>. the Adnumstration und the pubhc 

Simc ^^tJuniahihiy is. in pan. asscneJ by vontrol oscr funding, NAGH's hmicet 
should be separated I'om NAFP's. Stt hm^ ss XAOB obtains a pcfi'cntagc vf a <p<Menti4{h 
fap;d{y expanding J NAFP hud^-et. there is no way fot elected otfictaJs to adequately e*en 
ttverstght The roie of SAOB in irlatiun to NCHS. the Department n! Fducatton or anv other 
h^xiio crraied to kwctscc progress toward n*itiunal g<ials shouM be caret uUy considered b;. the 
ippntpnate House and Senate ccmimiliees and Ihe AdnunisEraiion befuir NAFP »s cxp.tnJeii 

]n xum. NAGB'v plans in raputly c){p:md ^AF;? »rhout adequate constder.ttum iif the 
etfcL'ts of the expansion or the proper role ^^f assessment m educatitmal ret<tn7t are UangeriHJs 
Neither Congress ctir the administratuin should aiftwt them to proceed withcnit careful rev »eu 
ami conswJeratton Similarly, the Oovemors should not support the use of NAEP for 
measunng progress lowanl nauonaJ goals without first clanfying the goal it and the role ot 
as-sesstneni m achieving them and then dciermtning the details of measui^ent Specifically 

- NAFP shtwid not be expanded to allow more frequent or CJttcrisive tciUtig or moxe 
detailed compar>^ons at least unu! compleuon of the tnai assessments of 1990 arwi 1992 ami 
the independent evaluation mandated in the Act TTheii. Congress, ihc Adnfiinistrauon and the 
Ciovemors must w^etgh carefully the potentially hannful effects of mote extensive testing and 
companions and ascertain that the dangers do not omwpjgh any possible benefits. In any 
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rvcnt. expansion of NAF.P mu« be subsrqucm and subordinate to the establishment of 
n^^uonai ^iMis anj not alto»^d loiiictdte a nattcmai coFriculum 

• NAKP should be durcccd to spend a ngnificani pomcm of its budget on developing 
and pjU>itng peilurmancc bawd 3S5rv\mcni& (inciodiRg ic&is and ponfolios). Such research 
and dcvcl4>pmcnt should be pianrvcd cansfuUyito coofdmaw wirh siaie projects smrb as those 
undcruay m California, Connecocut and Vcnpont. to de^-clop perfonnancC' based r^sessmcniN. 
a*. ns pn>jccts undertaken hy kval education authonues or («hcT govemtnenial or pn vatr 

- Conj^rcNs and thi Adiritnistrjium should cvm>kicr brparaung NACiB fundmg fnim 
NA! P f umlmj; ^fwl t arr^ully i on>idcr ihe fulurr wir of NAGB m rci.mon to other agencies 
and Kxiio 

Wjt .ipptfNi.sff v<Mjr jifrntum to the \c mnt importani in^^uvv af>d hx>k !ors»'afd f<i 
-Amkif}^ y^V,h \KMt in ihv ctU»n Iit jvhicvc ^cnu^iKT jnd lasting frt<>mis jrt ihe qii;thfv of p»h)K* 

P'.t jse Jcri tree i<> anv t>l u\ d \<\u hixe any ijue\t{onN or nrrd turthcr 
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HEARING ON NATIONAL TESTING: PROS AND 

CONS 



THURSDAY, MARCH 14, 1991 
House of REPRESwrrAnvES, 

SUBCOMMFITEE ON ELEMENTARY, Sm)NDARY, 

AND Vocational Education, 

COMMXTrSE ON EDUCATION AND LaBOR, 

Washington, DC 

The subcommittee met, pursuant to call, at 10:10 a.m.. Room 
2175, Rayburn House Office Building, Hon. Dale E. Kildee [Chair- 
man] presiding. 

Members present: Representatives Kildee, Hayra, Owens, Un- 
soeld, Roemer, Goodling, Klug, and Petri. 

Staff present: Susan Wilhelm, staff director; Andy Hartman; 
Lynn Selmser; S. Jefferson McFarland; Damian Thorman; Jack 
Jennings; Katrina Kelley; June Harris; Carole Stringer; a^id Beth 
Buehlmann. 

Chairman Kildee. The Subcommittee on Elementary, Secondary, 
and Vocational Education will come to order. 

As a former schoolteacher, I usually start right on time, so I 
apologize for the delay this morning and appreciate the witnesses 
bearing with me. 

The subcommittee meets today for the second in a series of hear- 
ings on educational testing. Yesterday, we heard from expert wit- 
nesses about the National Assessment of Educational Progress, the 
only federally mandated test A' student achievement. Todav's hear- 
ingfocuses on the pros and cons of national testing generally. 

There are several national testmg or ass^sment trains which all 
appear to be approaching the station at the same time. The first is 
guided by the need to measure pn^ress toward the national educa- 
tional goals which the governors presented to us. A second is 
driven by a perwption that a basic wtills test could give business a 
better indication of student achievement. A third promotes nation- 
al testing as a way to force educational reform. 

Todav^ hearing will attempt to address some of the questions 
raised by the cargo contained m each of these trains. That analogy 
may be stretched a bit, but we will use it anyway. It is the subcom- 
mittee's intention to carefully review the current educational test- 
ing system and to consider each of the positive and po^ible nega- 
tive consequences of a national test. 

I look forward to hearing from the manv knowledgeable and dis- 
tinguiahed witnesses we have here today. We are fortunate to have 
you with us. 

(1351 
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I call upon now my friend and the ranking Republican member 
of the full committee and the ranking Republican member of this 
subcommittee, Mr. Goc^ling from Pennsylvania. 

Mr. GooDUNG. Thank you very much, Mr. Chairman, and wel- 
come, all of you here this morning. 

We had an excellent session yesterday, and, as I indicated yester- 
day* one of my greatwt concerns at the present time is that all of 
the concerns in relationship to choice, national testing, national 
curriculum, national you name it, maybe out of frustration, say, *'I 
don't know what to do; things are bad, and ! don*t really know 
what to do, so let's just try all these kinds of things and see wheth- 
er something won't happen that will be better," rather than, you 
know, trying to improve education. 

I guess I would preface that statement by saying that I get pretty 
upset with the American attitude with respect to public education. 
Tne American attitude in relationship to Americans is always the 
same; we always put down ourselves, I talk to people all over the 
world, and they wonder why we are so critical of ourselves. 

We tend to lump all of public education into one little barrel and 
say there is something wrong with it, radically wrong with it, and 
v.e must do this, that, and something else. I looked at the list of my 
leadership, for instance. All of them graduated from public schools, 
I looked at the list of those who serve here on the committee, and 
on my side eight out of 12 were public school graduates, and there- 
fore 1 say they were successful before they came here, they have 
been successful since they have been here, and, yes, we have a lot 
of problems, but I sure don't want anyone to take the Wall Street 
Journal ^'National testing: No longer a foreign idea" as some kind 
of gospel, because it is about as misguided as anything I have read. 
Of course, it starts out by saying that national education goals 
seemed plenty radical when President Bush and the governors an- 
nounced them just a year ago. 

I didn*t know that they were ever considered radical. I thought 
we always were striving toward national goals. But then when the 
article really gets out into left field is, by saying "San Diego s edu- 
cation system should navigate by the same stars as Atlanta's,'* or 
**that a well-schooled youngster in Alabama should possess the 
same knowledge and skills as one in Minnesota.'' 

I mentioned yesterday that my State got off on the wrong foot. 
The former secretary of education in my State decided that we 
should test all students and then rank our schools in Pennsylvania 
according to the rests of those tests. Of cour^, I guess he was sur- 
prised to find out that a school district in western Pennsylvania 
was number one. Well, if it isn*t, it sure should be, because the per 
pupil expenditure is probably three times that in all but about 
three school districts in the Congressional district I represent. All 
the parentis are college graduates, many of whom are Ph.D/s. All of 
the parents insist from 6ay one, before the child opens his eyes or 
her eyes, that they are going to strive for excellence. Tlicv insist 
that their child strives for excellence. They also insist that the 
school strives for excellence. 

So my hope is, as we go through this, quote, reform process, and 
particularly the testing prcK?ess, that we are thinking primarily in 
terms of. How do I help this child with the information from the 
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test results that I now have before me?" I always try to convince 
my teachere that their purpose for testing was to see where they 
may have failed in printing the material that they wanted to 
present and where tW might find that they can help individual 
youngsters because of the results of those tests. 

I will repeat the story I told yesterday also of my first experience 
as a guidance rounselor. The superintendent mandated that I 
group all sixth gra^^rs that were moving into seventh grade. He 
didn t ask me whether I thought that was a good idea or not; he 
just said, ''Do it," and I started it and discovered that one class 
from Stewartstown appeared to alrwdy have been grouped, and 
when I asked the superintendent if that was true, he said, "Oh, no; 
you just have to understand, Mrs. so-and-so teaches to the t^ from 
dav one." 

Now I don't have any problem teaching to the test if there are 
wme specific goals that we want to reach and some knowledge that 
we think they should have, et cetera, et cetera Any teacher would 
teach to any test that they are going to give. But 1 want to make 
sure that it is a tool to help the youngster, and I want to make 
very, very sure that we don t compare apples with oranges, which 
is what our secretary of education in Pennsylvania tried to do. He 
is no longer, I might add, the secretary of education. He embar- 
rassed the governor, so he is no longer that secretary. 

Having said all of that, I welcome you to the hearing this morn- 
ing and look forward to your testimony. 

Chairman Kildee. Major Owens. 

Mr. Owens. Yes, Mr. Chairman. I ask unanimous consent to 
submit my written statement for the record and would just like to 
make a brief comment. 

I want to thank you for holding this hearing on the cjuestion of a 
national test and yesterday*s hearing on NAEP. This is an impor- 
tant issue and one which the Subcommittee on Select Education, 
which I chair, will also be examinii^ during its consideration of 
the reauthorization of the Office of Educational Research and Im- 
provement. 

The Department of Education, for example, has proposed to 
rename the Office of Educational Research and Improvement to 
the Office of Assc^ments, Statistics, Research, and Improvement, 
and I fear that these kinds of efforts will subordinate the more ag- 
gressive efforts that are needed to put the latest research in the 
hands of practitioners and to use our substantial knowledge base to 
improve education rather than assess. The Department's new 
motto, it seems, is going to be, **Ik)n't just do something; stand 
there and assess,'* and I think we have to take steps to guard 
against that. 

Thank you. 

[The prepared statement of Hon. Major R. Owens follows:] 

Prepared Statement of Hon. Major R. Owens, a Representative in Congresb 

FROM the State of New York 

I want to thank the Chairman for holding this hearing on the question of a na- 
tional test and yesterday's hearing on NAEP. This is an important issue and one 
which the Subcommittee on Select Education, which 1 chair will also be examining 
during ita consideration of the reauthorization of the OfTice of Educational ReBearch 
and Improvement. 
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Though some of its proponents may be »Jt arly sincere, I cannot help but view this 
new wave of interest in testing with great ii. ipicion. I fear it is yet another effort by 
the White House and the right-wing to substitute symbolism for substance in educa- 
tion poIi<7 making and desired to divert our attention from the ui^ent need for a 
gireater Federal investment in our children s education. 

We have seen this, for exampie, in the President s propc^ed budget for OERI for 
fiscal y^r 1^. The department is prop(»ing to rename the OfHce of Educational 
R^earch and Improvement to the Office of Assessment, Statistics Research, and 
Improvement. Aggressive efforts to put the latest research in the hands of practi- 
tioners and to use our substantial knowledge base in this area to improve the educa- 
tion of our children would be abandoned in favor of the passive role of mc ^!y as- 
seeing the status of education in this country. The Department s new motto, it 
seems, is don*t just do something, stand there— and assess. 

As they sit in overcrowded, dilapidated cJassnxims, children in New York already 
spend an average of four weeks every school year taking various standardized tests. 
There is something ridiculous about the idea that what they most need now is to 
have another, still more exotic test shoved under their noses in the name of holding 
them accountable for their performance. Our preeminent concern at this moment 
should instead be with holding politicians and policy-makers at every level account- 
able for their consistent failure to give our children and our schools all of the re- 
sources they need to succeed. 

Chairman Kildee. Thank you. 
Mr. Petri. 

Mr. Petri. Thank you, Mr. Chairman. 

I think we are having a hearing on an important subject, and I 
am looking forward to the testimony of the panels that we have 
today and to learning, particularly with the point of view of doing 
something, if we do it, that will improve education rather than 
doing something just for the sake of doing something in the vague 
hope that it might make things better, when the reality might be 
that it might make things worse. 

But let's proceed and learn and see if we can do something that 
is useful for the young people in our country. 

Chairman Kildee. Thank you. 

Mrs. Unsoeld. 

Mrs. Unsoeld. Thank you, Mr. Chairman. 

I, too, am looking forward to hearing the testimony, because this 
is probably a topic about which I know the least with which we are 
dealing. So welcome, panel, and the other experts. 

Chairman Kildee. Mr. Hayes. 

Mr. Hayes. Thank you, Mr. Chairman. 

I do not have a prepared statement. I do want to commend you, 
as others have done, for the scheduling of this important hearing, 
and I want to commend the witnesses for giving of their time to be 
here with us. 

I have some very deep-seated concerns about what I see as an 
effort to almost privatize the public educational system, which will 
diminish the opportunities for the economically disadvantaged and 
minorities from getting an opportunity for education. I am a little 
concerned that testing may be a step in that direction, and I want 
to be here and be a part of it, and I am just sorry I wasn't here at 
the beginning, but, as you well know, when you nave five subcom- 
mittees you serve on and two main full committees, we have all 
found out it is impossible to be at two places at the same time, 
much less three. So I just had to be somewhere else, but I am here 
today and expect to get much out of this hearing. 

Thank you very much. 
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Chairman Kildee. Thank you, Mr. Hayes. 
Mr Roemer. 

Mr. Roemer, Thank you, Mr. Chairman. 

I would just like to commend you and the ranking Member, Mr. 
Goodling, for holding this hearing. As a new Member, I am excited 
to be here working on education. This committee was my first 
choice. Education is the number one goal on the home front to ad- 
dress. Business people who are involved in educational efforts 
across the board in my community, come into my office to talk 
about improving education, 

I would like to welcome the uistinguished panelist, Mr. Brock, 
back to the Hill, and also the namesake. Governor Romer. I think 
we still have a couple of Roemers in the Democratic Party. 

I assume you don't have any plans to announce anything today, 
Governor Romer. 

Mr. Romer. I think the record should show that the majority of 
Romers have cht^n to not switch but stay and fight. 

Mr. RoEMER. The bipartisan Roemer efforts to improve educa- 
tion. 

I am excited about the panefs testimony and would just like to 
urge that we keep our focus on the long term and not the short- 
term political gain of coming up with an idea that looks good. We 
need to help more students who are coming out of school who want 
to enter the work force in a competitive economy. 

Thank you, Mr. Chairman. 

Chairman Kildek. Thank you very much. 

Now we will go to our first panel, consisting of the Honorable 
Roy Romer, chair of the National Education Goals Panel, Governor 
of Colorado and the Honorable William E. Brock, well known here 
on the Hill, a former member of the Senate, former Secretary of 
Labor, with whom I had the privilege of working— productively 
working, too— and chairman of the Secretary's Commission on the 
Achievement of Necessary Skills; and Gregory R. Anrig— Dr. 
Anrig— president of Educational Testing Service, Princeton, New 
Jersey— my three children who are now in college back in Michi- 
gan have sweated through your test a couple of times— and or^ 
who brings great credentials to the panel here this morning, 
known to myself: Trevor Sewell— Dr. Sewell— acting dean, CoIIl^w 
of Education, Temple University. 

Governor Romer, you may bwn, 

Mr. RoBiER. May I use the chart? Is that permissible hore, Mr. 
Chairman? 

STATEMENTS OF HON. BOY ROMER, CHAIR, NATIONAL EDUCA- 
TION GOALS PANEU DENVER, COLORADO; HON. WILLIAM E, 
BROCK. CHAIRMAN, SECRETARY'S COMMISSION ON THE 
ACHIEVEMENT OF NECESSARY SKILLS, WASHINGTON. DC; 
GREGORY B. ANRIG. PRESIDENT, EDUCATIONAL TESTING 
SERVICE, PRINCETON. NEW JERSEY; AND TREVOR 8EWELU 
ACTING DEAN. COLLEGE OF EDUCATION, TEMPLE UNIVERSITY. 
PHILADELPHIA. PENNSYLVANIA 

Mr. RoMBR. I wanted to use this chart. I know we need to be very 
strict in our time limits, and I will be. 

li.i 
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First, let me give you the power of this issue. As chairman of the 
National Goals Panel, 50 governors have agreed to hold themselv^ 
accountable each year for 10 years on the national goals. It is this 
panel's obligation to frame that report card and have it published 
between now and September. & our obligation is to take each of 
those goals and set specific questions that 50 States will answer, 
and that is real power, because the way you frame the question 
will set the agenda for those governors over a 10-year period. So 
the power of this panel is veiy, very large. 

Now, we have both a short-term and a long-run assignment. 
Short term, by September 1 we need to have a report card, and we 
will give it, but it is obvious that we do not have the data, the indi* 
cators, available in this Nation, or even State by State, to give an 
accurate picture. Therefore, our second assignment is long term, to 
develop an assessment system for this Nation to where we can 
know where we are but, more imi:K)rtant, where we can reform our 
system to reach where we ought to be. 

Now, that, therefore, is on our agenda immediately, and I would 
like then to share with you very oriefly s^ome thoughts we have 
upon the issue of a national test. I will give you my one-line conclu- 
sion. I don*t think we should have a national test; I think we 
should have national assessment system, and I would like to give 
you the outline for that system. 

But first, I found it is "» portant to talk in language that people 
on the street can understand, and I would like to give a couple of 
illustrations that point out what testing is to me. I happen to be a 
pilot, and when you are getting a pilot s license, it may take you 46 
hours to do it or it may take you 36 hours to do it, but whatever 
time it takes, there is a set of knowledge and skills, a body of 
knowledge and skills, you have to acquire 

I don't think any of you want to fly with an above-average pilot. 
You don't want a pilot who isn't at the top end of the bell curve. 
You want a pilot who has acquired certain knowledge and certain 
skills to master the craft. Therefore, I used to run a flight school. 
What do you do? You have an identifiable list of skills; you have a 
program of instruction that relates to that, and when you test, as 
you do, you are obviously using it for the purpose of assisting that 
student to know where he or she is in reference to where he or she 
ought to be. 

So I think just to keep in mind, first of all, that testing in its 
most important function, I think, is instructional, it is diagnostic, 
and if we are thinking about creating a national assessment 
system, I think it is a secondary benefit to know where we are in 
terms of a sample as a Nation or a State. The primary benefit of a 
test IS— can it be a part of the instructional tool? Can it In? a part 
of the reform of the system? 

Therefore, I just want us to begin to think about, all right, if we 
are going to be a pilot, there is a certain set of skills that we need 
to know, and it is obvious you can't test unless you !.now first what 
the skills are. 

Therefore, let s turn that from piloting to math. W ^en 1 go into a 
school, I often see the periodical chart of chemicals I think some- 
times we ought to take it down and put up this kind of a chart, and 
it is a chart on math, and every student in school can look at it 
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and say, "Wait a minute; at the twelfth grade, I need to know and 
be able to do the foiiuwing things: At the eighth grade, I need to 
know and be able to do these things; at the fourth grade, I need to 
be able to know and do those things.*' 

You know, it is kind of like a medicsl CAT scan. I go to the 
d(Xtor and take a CAT scan because there is an assumra level of 
health, and the CAT scan tells me v/here I am in reference to that 
and then says, through his prescription, ''Here's how you become 
well." 

I think educationally, a student in the fourth grade, the eighth 
grade, of the twelfth grade needs to have a clear understanding of 
what health is educationally. You need to know what the standards 
are you need to shoot for tc be employable, to be a successful citi- 
zen, and that is our first job, to arrive nationally, through a con- 
sensual process, at standanls of achievement. Then we will obvious- 
ly find in math that there is a range of skills, and somebody may 
be on this end of the range and be a technician* and somebcKiy may 
be on that end of the range and be an engineer, but if you are 
going through that system and you are at this level, you need to be 
forewarned that you are ill educationally, and you ought to take 
corrective measures, because you are not going to be a functional 
citizen. 

So, again, I would like to move from piloting to math and say we 
need to think of testing ^md what it does most of all for the individ- 
ual student and the parent of that student, because if we get that 
message across— namely, this is where you ought to be, and this is 
where you are, and we can make a change right now to get you on 
to the right track — that is the most benefit of testing. 

Let me then turn to the question of, should we have a national 
test to do this, or should we find a better way to do it? And I will 
conclude with this. When I took this assignment as chair of this 
anel, I tried to identify what I thought was the most important 
arrier to accomplishing its mission, and I concluded that it was 
the ideological conflict in this country about local control for edu- 
cation ana national goals, national standards; that is the conflict; 
and I said we ought to avoid that, we ought to avoid polarizing po- 
litically, and also substantively we ought to analyze what gives us 
the best product. 

I looked at a number of alternatives, and I came to the conclu- 
sion that we ought to try what I would call a cluster method, and it 
is going to individual States^ and let s assume that we are again 
dealing with fourth grade math; we are going to have to devise 
some new instrumentalities, because there are a lot of test instru- 
ments out there that, frankly, don't give us the right kind of as- 
sessment of the skills that we iit-ed to have. 

Therefore, if we are going to create some more authentic a-sst^ss- 
ment instruments of fourth grade math, including higher order 
thinking skills, there is no sense in our doing it individually. State 
hy State. I think we can save i^ome money by getting some Statt^s 
together and working together like an interstate compact. 

Let s say you had six States that are brought together either by 
their educational philosophy, geography, or whatever, and they 
form a group — call them Group One— and they develop together an 
assessment instrument for fourth grade based upon their particu- 
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lar approach- Let s assume that there are four such groups in this 
country. There is a way that that can happen and we can tie them 
together through a national standards mechanism, and that is as 
follows: to create a set of national standards on fourth grade math. 

Namely, we would arrive consensually at what it is a person 
ought to know and be able to do as a fourth grade math student. 
That, I think, can be done, and I think it needs to be done, and it 
ought not be done from the top down; it ought to be done consensu- 
ally from the bottom up, and there are ways to do that. But we 
arrive at what it is that a person needs to know and be able to do 
as a fourth grade math student as a set of standards. So that then 
becomes a— let me just call it the standante box. We agree upon 
that consensually in a national assessment system. 

But then we turn to these individual groups and ^y, "You create 
your own assessment instrument, but let's calibrate it to these 
standards so that when you are all done we know what a student 
who takes the test in one area would be as compared to all other 
areas/* and that can be done. England has a similar system. They 
have about five or six different panels that you can go to and say, 
**I want to use your test." Those panels are informally calibrated to 
each other. 

Very briefly, I think we can put together a cluster system in 
which we have the best of both worlds. First, we arrive at a nation- 
al consensus on what the standard is we are reaching for; we leave 
it to the local district to decide instructionally how you get that 
done; and then we share this business of assessment so that we do 
not have one test nationally dictated from the top down, but that 
we go to the individual States and say, ''Cluster; group together; 
use your own creativity; and then let's calibrate you through a 
standard box/* And the way that calibration can he done is, you 
just simply create an anchor test which, for those who are experts 
in the field, will work quite well. 

Now let me stop at this point and say, the reason that I throw 
out this suggestion^ the panel has not yet accepted this as the way 
to go, but we are doing a lot of discussion about it. I think that it is 
a way this Nation can get together with this polarity of, how do 
you keep local c ntrol of education and local ingenuity and creative 
ity and, at the same time, have some national direction as to what 
it is we ought to do in terms of levels of achievement and perform- 
ance? 

Thank you. 

[The prepared statement of Hon. Roy Romer follows:} 
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Mr. Chairman, mtuftb^rs of the coiwitta^, thank you for the 
opportunity to testify before the Eleinentary* Secondary and 
vocationai CducatioD Subcoflvnittee of the U.S* House of 
Representatives Committee on Education and Labor » The topic we ar# 
discussia9* the pros and coos of nationel education testing* is a 
vital one. 1 am pleased to take part in this debate «rith such 
distinguished conea9ue5 as yourselves and my fellow panel members. 

I have the privile9e to serve as the co-lead Governor for educarion 
for the National Governors' Association and as chairman of the 
National Education Goals Panel. I wont to make clear r hovever . that 
I am here today in my capacity as Governor of Colorado and am not 
speak inq in any formal way for NGA or the Goals Panel, The ideas I 
will discuss have been* and will continue to be debated by the 
Hovernors and by the Goals Panel, They are creating interest and 
exciteisent. but neither the National Education Goals Panel nor the 
NGA have yet adopted a formal policy on the issue of national 
education testing. I want to begin by giving you a very brief 
background on the National Education Goals Panel. National 
Education Goals were established to provide a co^on framework and 
vision for education reform. As GoverDors* w« wanted to be cleai 
fhat wp were committing ourselves to making education a priority in 
©arh of our States and were willing to be held accountable for this 
commitment , 

Th^ goals themselves have been criticised as too ambitious. Ii* my 
judgment, they accurately identify what out reach must be to make 
the kind of educational progress that is reguired to semrp our 
future. The goals are not gospel. They are not perfect* But thf»y 
do provide a common vision, a common way to organize reform issues, 
and a common language. 

t,a5t July, the National Governors" Assoc iai'^.i adopted a policy 
establishing the National Education Goals r^anel to oversee the 
development and implementation of a national education progress 
reporting system. Beginning in September IQ^l. the Panel will icsue 
an annual progress report to the nation on our progress toward 
achieving the oati'*'.al education goals. 

The Panel's mom involves two tracks: one short-tesm and one 
long term track. The short term agenda is to produce a rrffrtihle 
report to be released in September 10<^1 . In working toward ^his 
deadline^ the Panel is aware that great harm can be done by asking 
the wrong questions or by trying to force existing data into new 
forms. Accordingly, the Panel is committed to a thoughtful and 
rowprehensive approach over the long term, even if it means leaving 
some blanks in the first report. 
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Our lanq-t#tf« Agenda involves encouraging the development of An 
assessment fty&tem that will not only measure our progress towsr<S the 
goals, but that will help us restractuie the education syste«" to 
arhieve the goals. 

In an effort to take advantage ot the expertise that esists m the 
country on these issues, the Panel established sis working groups to 
91 ve advice and policy options to the Panel abouf its woi k . both 
Short terw and lonq-terw. These groups are in the process of 
developing creative approaches for measuring our progress in each of 
fhe six goal aieas. A list of those working groups is attached io 
this testimony. Their rpromjnendat ions to the Panpl will be released 
late this month. 

With tespect to goal tB. increasing student achievement, the Panel 
has identified thtee waves of action and effor». that will need to be 
mobilised if we are to make substantial progress toward the goalR by 
the year 2000'- 

• Defining specific standards of student achievement <what a 
Ktudent should know and be able to do): 

• Pesiqninq an assessimenf system tha*" will allow us t^ 
acrurafely and authentically measure our performance 
against these standaidsk.* and finally. 

• Moving t^he eduf'ation system (-}ose fhe gaps hetwe^^n 
rnrrent levels of performance and the standards. 

These * ^skn lie along a rnntmuum from national fo local effoit. I r, 
*he Panel's judgment, national goals and national standards for 
student performance are appropriate and necessary. At the other end 
of the continuisn, we believe chat the design and implementation of 
specific reform strategies must remain locally based in the best ot 
our traditions. In the middle* the assessment challenge provides an 
opportunity for Joint action. Let me discuss each of these in turn. 

Stdil<lAI^- need to begin with detailed goals and objectives for 

learning. These objectives must be uniformly high and must reflert 
what future citieens of our nation will need to know and be able to 
do. Given the diversity of reform strategies, an overall national 
educational stsndards framework is needed to provide the targets at 
which all would aim their efforts. 

Discussion regarding the process by which these jitandards ran he set 
has jvist be^un. The following elements have been identified as 
critical t*"* this process: 

• There must be a key role for Governors and the states, 
which still have primary policymaking and funding 
responsibility for education. 

• The process mu&t be highly consensual, participatory and 
inclusive. Broad consultation with all segments of ouf 
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population is th# oniy %fay to build th« commititmnt of all 
Americans to the hard job of educational change. In fact, 
the standard-s^tl* tng process can itself become part of the 
renewal process if it represents a shared clarion call for 
higher espectations and higher standards for all students. 

♦ To make sure the standards we set are competitive, we will 
need to use as a henchaark the best standards in the world. 

• The process would set standards, not prescribe hov the 
standards are to be achieved. Consnon standards need not 
and should not give rise to a national curriculum that 
dictates to teachers what and how they must teach or that 
1 imi ts the choices of school districts or states in 
selection of textbooks or instructional mater ia Is. The 
standards would be a description of the concepts, 
knowledge, and skills that students should master^ not a 
recipe for achieving those standards. 

MfiftfiSM&t* Tests should have the capacity to support reform and 
not juftt -neasure present performance. Tests also should hftve the 
capacity to measure the skills and abilities student- till need r.o 
know . 

1 flkm concerned that many of frhe assessments we use nov don't 
authentically measure higher order thinking skills. We reiy on 
standardised tests that tend to drive curriculum and i»i3truction 
toward rote memorisation and the discovery of s single "right" 
answer. I am also concerned that the tests on which we rely to 
provide national and state samples of performance dont give the 
kind of feedback individual students need to help them and their 
parents judge the adequacy of their performance against national 
standards. These tests are useful and I support their continuation, 
hut for assessment fo stimulate better performance by students, thfi 
test ultimately has to be important to each student. 

The notion of testing all students rather than a sample of students 
raises legitimate concerns about local control of education. There 
is a fundamental tension between the compelling need for national 
goals and standards and the tradition and strength of local 
control, «e know that the most effective schools are those that are 
managed at the school level. ki« draw strength from the diversity of 
approaches that local control makes possible. 

Therefore, I do not advocate a single federal test . Instead. I am 
interested in esploring the alternative of a national esamioation 
system. This system would reflect not only the voluntary 
participation of states, but the powerful combination of their 
individual expertise. The system would be composed of several 
examinations anchored by a coomon set of national standards » 
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States, which already devote enormous resourrcs to testing, could 
wotk in clusters to develop »nd implement examinations. Each 
cluster's examination »wuld be designed to assess the range of 
knowledge and skills encompassed by agreed upon national performance 
standacds. 

riusters miQhf^ be formed around different interests and states would 
be Cree to decide which cluster examination was most appropriate for 
their use in any given subject* The interests of clusters could 
evolve over time and states would be free to change th*»ir 
associations . 

Through an anchoring or calibration procfc^^tare . a means for equating 
student peifoimance across clusters could be devisad. The key to 
this process is a commonly agreed upon def inition of what students 
shotild know and be able to 60, Th#»^ examinations cannot* strictly 
speak ing* be designed until national standards have been developed 
and adopted. Kowevex , we can begin work on the process of 
designing and evaluating examinations formats. We can develop a 
format which will work will in the anchoring System and begin to 
experiment with different methods of calibration. 

In suggesting this approach* I do not wish to minimise the degree of 
technical work that will be called for nor the time required for 
developing the system. However « at this point, it appears that this 
type of assessment system is both technically feasible. and 
essential to meeting our national reform objectives while retaining, 
and even strengthening^ our tradition of local control. 

iift»4tfitfcinff Chm^n in Svetg». The changes that this type of 

assessment system could help drive includes 

e Schools will offer challenging learning opportunities for 
all students, not )ust those who are col lege -bound. For 
most of the twentieth century, America has had two systems 
of public education* -One designed to train a small group 
for management and leadership and another to prepare the 
remainder for the routine work they would do as adults, vie 
now recognise that a work force dominated by individuals 
who are not challenged to achieve higher than eighth-giade 
skills is uncompetitive in a greatly changed- -and changing 
world economy, 

e Learning environments will be structured to encourage and 



reward student effort. 



Parents will have the knowledge and cnotivation to be 
effective partners with the school because they will 
understand the Bkills and abilities their children must 
have to prosper economically and how their children are 
performing in relation to this standard. 
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* The meane by «#hich asiess pertonnance will accurately 
track what we want students to know and be able to do. 
There is a co&sensus emerqinq that the new bas ic skills 
include rr it ical thinking* information manaqem^nr * 
interpersonal and comtnunicat ion skills^ and problem 
solving. Not all classrooms and very few tests presently 
focus on these skills. 

• Assessment of student learning will have consequences. I f 
employers and colleges participate in setting the standards 
on which the examinations are based, and agree to take 
per formance on the examinations into account when deciding 
who will be admitted to college and who will get hired at 
what pay level, then millions of students wili, for the 
first time, siw a direct connection between their 
pet formance in school and their opportunities in life. 

I am a pilot. The system l experienced in obtaining my pilot's 
license provides a useful model of the kind cf system we ate aiming 
toward. When I began my course of study r X clearly k. '^w the 
competencies and understandings that Z would have to demonstr«.te to 
obtain the license. And I directed my effort at achieving these 
understandings and competencies. Some came faster and easier than 
others. The length of my course of study did not depend on an 
arbitrary number of seat hours* but on mastery of the required 
skills* I established my conmand over the required subject matter 
through a demonstration. The test therefore related directly to the 
subject matter I was being tauqht-^and to the skills I need to pilot 
an aircraft safely. 

An examination like this one. that can organise and motivate effort, 
that can accommodate individual differences in learning ratem and 
stylesr that provic real consequences for the learner and that 
establishes standards of performance that all students are expected 
to meet has much to offer as we reflect on the role assessment can 
play in education refom. 

I welcoeie a costittuing debate on these issues. As chair of the 
{tational Education Goals Panel and co-lead governor for education 
Cor NCAr I hope and intend to have a close working relationship with 
this Committee. 

Tbank you again for the opportunity to participate in this hearing. 
I would welcome any questions or conmeots from members of the 
subconvittee. 
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Chairman Kildee. Thank you very much, Governor, You are a 
good teacher. I used to use the blackboard myself, and I found it a 
very effective tool there. I appreciate your presentation. 

Senator Bill BrcKrk. 

Mr. Brock. I am going to begin by departing, Mr. Chairman, 
from my initial thought to say that I wish every State in the coun- 
try had as gwd a governor as Roy Pomer. I really appreciate the 
leadership he is giving to this is^ue, and I think it is fortunate for 
the country that he is in the chair of this particular committee. He 
is articulate, but he is thoughtful, and he is absolutely right. I sup- 
port, I think, almost every word that he said. 

Let me just draw back and try to look at this thing with a bit of 
perspective. I have been worried about education for a very long 

geriod of time, but it got prettv specific when I was the United 
tates Trade Representative and I had to wrestle with people who 
kept coming in and saying, **Why are we exporting our jobs? Why 
don't we keep them here? Why don't we have the ability to defend 
ourselves against people who are outproducing us?*' And the first 
complaint 1 heard was that all these competitors were coming in 
with slave wages. Well, that is teloney. We are now not the high- 
est wage country in the world, and we are getting beaten by people 
who are paying higher wages than we are. 

What is the difference? The difference is that we have slipped 
behind in giving our children and our workei^ the tools to be as 
productive as they are capable of being, and that became reinforced 
when 1 took the job as Secretary of L^bor, and we did this Work- 
force 2000 study which said, in effect, that unless we change pretty 
quickly we are not going to be competitive in many areas at all and 
we will have to compete on the basis of low wages. 

Then I went to cochair with Ray Marshall this Commission on 
the Skills of the American Work Force. We published last year a 
study called America's Choice: High Skills or Low Wages," and we 
came to the conclusion, based upon fact, not opinion, that this 
country, without understanding it or really consciously deciding to 
do so, has b€^n trying to compete with the rest of the world on the 
basis of lower wages rather than higher skills for alM)ut 20 years. 

Real wages in the United States for 70 percent of the American 
people are lower today by 12 percent than they were in 1970. Now* 
uuggone it, that has to tell us something. Why is it that, after 
something like eight years since the ''Nation at Risk** was pub- 
lished by the President's Commission back in 1983, we have talked 
almost ad nauseam about education reform but nothing radical has 
happened? We have been trying to make changes at the margin. 

Why is it that most people know the urgency of education reform 
but don't think it applies to their school? Why is it always some- 
body else's problem? 

One of the reasons is that we really don't have very good infor- 
mation on what we are doing. One of the reasons is that we get 
caught up in the shibboleth of politics when we say, "Oh, my gosh, 
we can*t have a national test; that violates local control/* 

Let me describe what a national test is. An SAT test is a nation- 
al test. Not everybody is required toiake it. College Boards are a 
national test. Companies give tests; \GE gives tests all over the 
United States; IBM does; they don't vary; they are national tests in 
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a way. The U.S. Army gives a test to every student; it doesn't 
matter where they come from, what their race or creed is; they all 
take the same test. Is there something unconstitutional about it? 
No. What it does is tell the Army who is qualifled, or, more impor- 
tantly, it tells them how to train them to he qualified. 

We just went through a war. The Patriot missiles may have been 
the most important instrument in that war. Well, you can't fire a 
Patriot mi&aile until you have had, first, a high school education, 
and then, second, full basic training, and then, third, a full addi- 
tional year of training in that specific skill. 

Well, it seems to me that we need to quit aiding about who has 
got control of this thing and argue about what we do about a prob- 
lem. The problem is that our children are not getting an adequate 
education, and our parents are not being given information on 
which to base their involvement in the educational system, and if 
we are going to change that we can't argue about national assess- 
ment. As Governor Romer says, we can develop a s)rstem of nation- 
al standards, States can have different methods of testing if they 
want te, so long as there is a way of relating that test score of their 
test to the national standard, because we do need to know, and par- 
ents need to know, and kids need to know, whether or not they are 
able to compete globally. 

One of the reasons there needs to be a national standard is be- 
cause we have to benchmark ouiwlves against the world, not 
against Texas and Arkansas and Tennessee. We are not competing 
there, we are competing in \he world, and if you look at the pur- 
pose of the classroom test. Governor Romer was absolutely right, 
the primary purpo^ is to diagnose the problem and let the individ- 
ual have some understanding of what they have to do to solve it. 
The purpose is to provide information to the student, the parent, 
the teacher, so that they can act to solve the problem. 

If our medical system were acting like public education is today, 
you would go to the doctor, and he would treat you and then wait 
for six months to di^nose you, and then, when you didn't respond, 
he would say you were the problem. 

Well, that is pretty dumb, but that is the way we do things in 
education, and it seems to me that the first step is to give a differ- 
ent tone to this idea of assessment. I like that word better than 
'*test," by the way. We are the m«3t overtested and under^ucated 
people in the industrial world. 

But if we will provide that information that Governor Romer 
sketohed up there» whether it is in maUi, or science, or in function- 
al skills, like reading a bus schedule, if we provide that informa- 
tion to an individual and let them know where they are in relation- 
ship to people around the world that we have to compete with, 
they will respond, 

lliose tests allow the student to respond, and they allow us to 
diagnose the student's problem. Not every student is going to be a 
balterina anv more than they are going to be an engineer, but each 
one has to know where they are on a scale of performance in a 
range of sublets so that they have a sense of what they are trying 
to achieve. We need those comparisons for the tochers; we need it 
for sdiools; we need it between schools and districts and States. 
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The NAEP does monitor national progress; it does not in any 
way measure individual students. We need to advance the curricu- 
lum, because it is not accurate to say that 50 percent of our 18- 
year-olds can't read at the ninth grade level and 50 percent of our 
IS-year-olds can't perform at eighth grade math level. 

What we are trying to do in the SCANS Commission is to re- 
spond to these problems by suggesting that there is a different way, 
we can begin to teach within a functional description, we can teach 
in a real world circumstance, and we can b^n to evaluate stu- 
dents like Boy Scouts measure people who are trying to become 
E^le Scouts, with a series of competency evaluations over a period 
of years. If you do that, then you are really beginning to give them 
the toob with which to achieve the level of progress that they have 
got to achieve. 

What we are looking at is the presibility of trying to have assess- 
ment certificates, if you will, or merit badges, if you want to use 
that term, that would deal with a whole range of different func- 
tional skills and use tht»e in the sense that they have to relate to 
the need or their ability to work in business but also to compete in 
the world. So they have to be internationally benchmarked. 

Let me just conclude with one additional point from this Com- 
mission on Skills that Ray Marshall and Ira Magaziner and I 
worked on with a bunch of other very fine people. The rasence of 
the conclusion we reached was this. Our system has to prepare 
young people by the age of 16 — b^ause that is when they can le- 
gally drop out of school in most States—to a globally based stand- 
ard. We said if we can get to that point, then they ought to be 
given four more years in what I call a GI bill for all students that 
would be paid for, in my judgment, by a 1 percent surcharge on the 
income tax. There are a lot of ways to do that, but that is not the 
i^ue. The point is that we ought to give them additional education 
beyond that in a whole range of fields, but we have given them 
enough schooling by that time to have a choice that they don't 
have today. 

Secondly, if they don't get to that point, it is not their fault, it is 
ours, so we ought to have a system of alternative learning centers, 
because not all students are going to make it in public schools as 
they are presently described that are available for any student that 
wants to drop out or choose an alternative method, so that we can 
get them to that globally derived standard so they can be produc- 
tive as human beings, and if we do that, then we are beginning to 
respond to their need and to this country s desperate need to pro- 
vide our workers with the skills and the tools to be productive 
internationally in the sense that we can compete with anybody. If 
we do that, we can eat anybody's lunch and we can start competing 
on the basis of higher wages and higher skills rather than low 
wages, which is a suicidal way to go. 

Thank you, Mr. Chairman. 

[The prepared statement of Senator William E. Brock follows:) 
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Mr. Chairman and members of the Subcommittee; 

I appreciate the invitation to participate in this hearing 
and discuss a very important issue. I have long held a deep and 
abiding concern about the need to improve the quality of American 
schools in ordor to prepare our young people for the realities of 
work. My experiences as Secretary of Labor and U.S. Trade 
Representative impressed upon me that our nation's economic future 
and intern<auic;.2l competitiveness depends on iriproving the 
countjry*s education system* 

The landmark report yprkforce 200Q . which I coiwnissioned as 
Secretary of Labor nearly four years ago, documented that the 
skills gap is constraining economic growth in the United States. 
For nearly two decades, the skills of our workforce have been 
inadequate and have contributed to a decline in real wages for 
most Americans and the widening gap between '^hose who go to 
college and those who do not* Moreover, it has diminished the 
United States' ability to compete in the global marketplace. 

This morning I want to relate the work of the Secretary's 
Commission on Achieving Necessary Skills, which I chair, to the 
debate on national assessment. I want to discuss how we are 
approaching the workforce skills crisis and how we think the isisiie 
of national assessment fits into the picture. Let me begin by 
sharing a few general, personal thou*^hts about national 
assessment . 

WHA T IS HATIO H AL ASSESSMEHT? 

First, I taink all of us who talk about national assessment 
should clarify what we mean by "national.** Alarm bells tend to go 
off whenever the word "national" is associated with issues 
surrounding elementary and secondary education. First, we should 
distinguish between federal and national exaras. Examples of 
national examination systems abound. 
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Is th© College Board, and their SAT and Advanced Placement 
exams, national? These tests aon*t vary by state « but the tests 
are not taken by all students- Are company esployroent teats 
national? Again, the tests given by corporations like IBM and GE 
vary by company, but they don't vary geographically. 

Our troops are now coining home after doing a remarkable job 
in the Persian Gulf- They offer a good example of what I'm 
talking about- While their high school educational experiences 
were all over the board, they all took the sa»e tests to get into 
the armed forces* The Army does not use different enlistment 
tests in Flint, Michigan and Chattanooga, Tennessee- The Arny is 
able to know how the young man or woman from Flint compares to tho 
new recruit from Chattanooga, 



Second, we need to understand the purposes and value ot 
assessment. As I see it, the assessment process in education can 
serve five main purposes: 

First, classroom tests can help guide instruction for 
individual students. Used early and often, they can help students 
~ and their teachers — identify problems and address them. If 
the way most schools use assessment today wore transferred to the 
medical world, doctors would prescribe treatment first, wait five 
months to diagnose the problem, and then blame the patient because 
he or s.>e is still sick. In the American classroom, we need to 
diagnose the problem first and then prescribe a cure and monitor 
the progress. 

Second, assessments and evaluations can be used to identity 
superior or inferior performance by teachers, schools, school 
districts, and states. Statistically valid samples are adequate 
for this evaluation purpose. 

Third, systems that have certain consistency to them can 
help us monitor national progress. The National Assessment ot 
Educational Progress (NAEP) is an example* 

Fourth, assessments can be used to influence curriculum, botr, 
how it is c?^signed and how it is used. Instead «f a test having 
no relationsh.^ p with curriculum, it can be used to shape what is 
being taught. Toda> . to a degree, Advanced Placement exams do 
this for the minority of students who participate. 

And fifth, assessment systems can certify the ccnpetency 
level of a student so an employer or college can make a reasonable 
decision about hiring or admission and placement. In a world 
where students often go to college or work far from home, these 
exams must be nationally comparable to be useful. The SAT and 
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Advanced Placement exams serve that purpose fcr colleges, but 
there is no similar sodel for eoployers. For vitose who are 
skeptical about assessment for certification purposes, a voluntary 
system that is of value to the students should not be a problem. 

SgftHS MP ASSBSSMMT 

That brings me to SCANS. Although wo have much more work to 

do, our recommendations on assessment will probably be designed to 

serve the last two purposes — influencing curriculum and 
certifying mastery of skills. 

SCAKS is based on the premise that today there is an 
insufficient connection between education and work. Students have 
little understanding about how school relates to their later work 
on the job. And they are not being taught the kinds of skills 
they need for productive employment in today's workplaces. At the 
same time, businesses arc not demanding skill'^id workers and they 
are not telling educators, parents, and students what kinds of 
skills they need. As a results th« value of the high school 
diploma has diminished. And American youngsters study less than 
their counterparts in other countries. 

SCANS is trying to forge a strong link between school and 
work. To do this, we believe a fundamental overhaul of the 
American education system is needed » The change we envision 
would help provide all students with the competencies they will 
need for success in the workplaces of today and tomorrow. 

Last summer the Commission on the Skills of the American 
Workforce, which 7 co^chaired, stressed che need to restructure 
school and work. This, we felt, is esp«:fciaXXy important for the 
so-called "front-line" workers, the majority of people who will 
not get a college degree. Building on the work done by that 
Commission, SCANS is now working to recommend specific elements of 
that restructuring. 

To begin, SCANS is identifying the specific skills people 
n<^ed for productive and meaningful employment. These will 
include the basics of reading, writing, and math, as well as more 
complex skills, such as the ability to communicate, work in 
groups, solve problems, and use technology. 

But how do w© get teachers to teach and students to learn 
these skills? SCANS is considering the idea of using a series of 
certificates that will accurately reflect mastery of specific 
skills. 

The certificates would serve the two purposes lacking in our 
current assessment system. They would provide a clear target for 
instruction and they would let employers know about the 
achievement of prospective employees. 
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For this kind of aasessiaent systeio to work, it must be part 
cf a comprehensive approach that includes curriculum, instruction, 
and student motivation. Schools could design curriculum to help 
students meet certificate requirements* And the certificates 
would provide a real incentive for students to work hard in 
school* 

It is important for mo to stress that SCANS is not talking 
about a series of multiple-choice tests. More sophisticated 
forms of assessment are needed to evaluate the acquisition of the 
complex skills needed in today's workplace. After all, how can a 
multiple-^choice test tell how well an individual works in a group"' 

Lauren B. Resnick, the Director of the X^earnxng Research and 
Development center at the University of Pittsburgh, chairs SCANS »s 
Assessment Committee* Dr* Resnick describes three broad types of 
assessment t.hat could serve a diverse population and recognize 
many kind<< of skills. They are performance examinations 
measuring specific tasks, reviews of portfolios of work done over 
time, and evaluating projects done individually or in groups. 

It is also important to stress that the certification systcr 
we envision would not be pass or fail. Rather it would measure 
various levels of achievement, in a similar way that the College 
Board's Advanced Placement exams have various levels. Our goal is 
not to identify failur^^ but to identify problems and help 
students succeed. 

SCANS also is not talking about a single national curriculum. 
While the certificates would be consistent nationwide, various 
kinds of curriculum could be used to help students attain them. 
But all students would be moving toward uniform, high-skill 
targets. 

For such a certificate and assessment system to work, we will 
need some kind of a governance mechanism. It won't work in a 
vacuum. An«i we must ensure that the skills we are demanding are 
up-to-date, reflecting today *s needs. 

To ensure that education is effectively linked to work, 
employers must have a strong voice in determining what skills an^i 
knowledge are needed. At tomorrow's SCANS meeting, we will be 
discussing whether an '^employer board** would be the best way to 
administer the program and issue the certificates. The board 
would be composed ol representatives from business, labor, 
education, and citizens* advocacy groups. It would be charged 
with setting objectives and criteria for certificates and 
establishing a system to assess them. 
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TAKIKG ACTION 

The ideas I have nentioned describe the direction in w*?lch 
SCAKS is going today* But it is important for ne to stress that 
our work is a work in progress. TOe 31 representatives of 
business, education, labor ^ and governxsent who serve on SCANS are 
reviewing all of these concepts* We intend to release an initial 
report in late spring and further recommendations later on. 

But the bottom line is that SCANS *s goal is to produce 
recommendations that will lead to a fundamental change in the way 
American schools prepare students for the world of work. The time 
tor rhetoric is over; the time for concrete action is here. 

Ny thinking on these issues has changed dramatically since 1 
served in this House more than two decades ago. But the world has 
also changed dramatically. The demands of the modern workplace 
are more complex and we face stiffer competition than ever from 
abroad, tfe can no longer be satisfied with tinkering around the 
edges of an education system that is failing to provide the kind 
of skilled people our economy needs to remain competitive in the 
years ahead. 

The SCANS mission i^ anbitious, but the rewards for our 
success will be substantiai. It will lead to a aore competitive 
and vibrant economy, better employment opportunities, and a higher 
standard of living for our children* 

I am pleased that you are taking the time to seriously debate 
the issue of national assessment. After years of considering how 
to improve our education system so it better serves individuals 
and the nation's economic future, I am convinced that some Kind of 
national assessment system is needed to meet that goal. 

/ # # 



5 



157 



Chairman Kildek, Thank you very much, Senator Brock. 
Dr. Anrig. 

Dr. Anrig. Thank you, Mr. Chairman. 

First, I want to commend you and the members of the committee 
for having such a hearing. 1 think what you are doing is shedding 
light on a complex issue where people are prepared to run with a 
simplistic answer, and I really admire the opening statements of 
you and the members of the committee on this issue. 

First, Mr. Chairman, I want to be sure to point out to you that 1 
am a graduate of Western Michigan University and 1 practice 
taught in Paw Paw, Michigan, and my three roommatf^ in college 
were from Flint, Michigan. That just goos to show how long I have 
been preparing for this hearing. 

Chairman Kildee. You have great credentials. 

Dr. Anrig. I come here today to speak in opposition to a national 
test. Such a test, in my judgment, is a well mtentioned but ill ad- 
vised solution. This may seem surprising to you, to hear this from 
the president of Educational Testing Service, but I take seriously 
our charter as a nonprofit organization to promote public under- 
standing of tests and of proper test use. I think a national test is 
not a good way to use tests and is not a proper way to use tests. 

I oppose national test propc^als for three reasons. First, embrac- 
ing a national test is like choosing a referee before you know what 
game you are playing. We first need to define what the educational 
goals mean in terms that are u^ful for instruction. I think the na- 
tional education goals were a very positive first step of the gover- 
nors and the President. But one of those goals says that by the 
year 2000 the United States should be first in student achievement 
in mathematics and science in the world. 

Now what does this mean to a fourth grader in Michigan, Penn- 
sylvania, or in the other States represented on this panel? And 
what does it mean to their teachers? What do we expect them to 
know and be able to do? We need to answer that question first, 
before we turn to the question of how to measure their progress. 
This essential first stage shouldn't be shortcut by letting a test pro- 
vide the definition. By its very nature, any t^ is simply a sample 
of knowledge and skills. If you use the test to define the curricu- 
lum, you will end up with a sample curriculum. That doesn't make 
sense and would not be good for education. 

A second reason why I oppose a national test is that, again, as 
throughout the period of education reform, we are bypassing teach- 
ers. Their feeling, in all of the surveys that I have seen of teacher 
attitudes towards education reform, by the Gallup Organization, 
the teacher organizations and others, is that reform is something 
that is being done to them and their students, not for them 

Now, if you taktr a look at what you are trying to accomplish, 
what we are all trying to accomplish, in reform, we are trying to 
achieve improved student learning. As Governor Romer so well 
pointed out here, you want to be sure that line is going up and that 
the pilot is not just average — or brain surgeon, just average— but 
really good in what they are doing. That is what the public is ex- 
pecting of us. The only people that can deliver that, with all re- 
spect to this panel and to us and others in this rcK)m~the only 
people that can deliver that are the teachers in the cla^room once 
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that door closes, and we are excluding those teachers from the 
process of reform. 

Some of the business leaders whom I admire very much and have 
teen very active in the reform movement have learned the hard 
way that competitive productivity means that you involve your 
work force in deciding what it is they do rather than dictating to 
it. We can't dictate to teachers and expect them to close the door 
and develop excellence in their students unless we give them a 
chance to be part of that action. I think that we are making a bad 
mistake at this point in not involving teachers more in a central 
role in reform generally and particularly in the issue of setting 
standards, determining learning outcomes, and deciding best how 
to asse^ and measure them. 

Finally, a national t^t simply won't work, and we have plenty of 
experience to show this. States and school districts have tried the 
testing route to educational reform, and it hasn't worked, and a 
recent report funded by the Ford Foundation said that the amount 
of effort put into testing right now, today, is 20 million school days 
per year and over $100 million in direct expenses of tax money 
right now. Thirty-eight States had statewide testing programs by 
the end of the 1970s— 1970s. Forty-seven States now have state- 
wide testing programs. 

What are the results? The National A^essment of Educational 
Prepress, which ETS is privileged to administer on behalf of the 
National Council for Educational Statistics— NAEP— indicates that 
since the 1970 s, during all this period of all this testing at the local 
nnd State level, we have experienced some modest gains in basic 
skills, some modest gains, and they are quite mcNjest, and only me- 
diocre performance at the middle and higher levels of achievement 
in reading, mathematics, science, writing, and civics. I see no 
reason to believe that a national test can do what all this effort 
and investment by States and localities has failed to do with all of 
the testing that has gone on up to now. The bottom line for the 
public is better learning, and we can't test our way to educational 
excellence. 

Now what can work? I believe Governor Romer in the National 
Goals Panel is on the right track, and I think the SC^ANS Commis- 
sion is heading in the right track. The national test would be the 
wrong track. We have a need for what the national panel refers to 
as a nationwide assessment system. Now what does this require? 

This requires, first of all, that we establish standards for all sub- 
jects at important transition-based points, based on a broad-based 
consensus that includes skills as well as knowledge and address, as 
the SCANS Commission is suggesting, employment relevant as well 
as academic outcomes for all students. 

Second, we should define what ih to be learned before deciding 
how to assess this learning, and we should focus primarily on out- 
comes so that teachers have the opportunity for flexibility and cre- 
ativity in terms of how to best achieve these outcomes. 

If you wonder whether that can be done, by the way, the Nation- 
al Council of Teachers of Mathematics and the Mathematics Asso- 
ciation of the United States have issued standards for mathematics 
instruction in the schools. This one publication, not thick, covers 
all of math from kindergarten through twelfth grade, highly re- 
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spected standards, without telling jwople how to do it. It can \^ 
done, and it has been done by that organization, and I hope others 
will do it as well. 

We should assure that students are taught what the standards 
call for. One of my greatest worries about a national test is, we will 
go off happily and issue a national test and we won't be sure that 
the youngsters are being taught what is called for in that national 
test. Now what is that in terms of fairness? That is not fair. That is 
not good test um. 

We should be sure that the students know what the standards 
are and are taught that, the teachers know what outcomes are ex- 
pected of them, and that schools know what goals they should 
strive for, and, as I have said before, 1 believe teachers should be 
significantly involvwi, along with others, in determining each of 
these. 

We should link local creativity and assessment with State and 
national indicators of performance — this is another part of what I 
appreciate of the govemor*s proposal here of clusters— and promote 
the use of a variety of assessment techniques that include student 
produced results based on group as well as individual effort. Most 
tests are designed for individual performance. In mast work, you 
don't work individually, you work with others, and we ought to 
find ways to begin to build that work habit into what we a^ess in 
education. 

We should vigorously guard against creating a new system to 
sort and label children, especially those bom into poverty who 
start childhood with educational disadvantages not of their own 
making, and. Congressman Hayes, I notice that you spoke to that 
in your opening comments. 

One guard against that is that we should gather and report indi- 
vidually identifiable student scores only when that information will 
be used to provide improved educational opportunity and/or addi- 
tional instruction to these students whose scores are to be reported. 
All other assessment should be conducted on a sampling basis to 
guard against test misuse. And, finally, we should use assessment 
p<^{tively as a basis for substantial incentives to schools and school 
districts and, for thc^ in need, sutetantial help to them. 

Now this doesn't d^rite a national test. What this describes is 
a national assessment system, and that, Mr. Chairman, is what I 
came here to speak to today. We need to build on the American 
traditions, not on those of foreign countries. We could just turn to 
Europe and Japan and say, well, let s do it their way, but we are 
not and never have l^n Europe and Japan. Well, at one time we 
were partly Europe. But we don't have a centralized system, and, 
thank G<xi, we don't have a federalized system, and we shouldn't 
try to make one. We have a diversity of student population, a di- 
versity of strengths, of States, and a commitment to equal opportu- 
nity for all kids. That is something we should preserve, and we 
shouldn't move to a test in a way that will defeat that. 
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I have identified some existing models in my written testimony 
that could provide building blocks for a nationwide assessment 
system. It can be done in parts, it already is being done, and I hope 
we show good sense in doing that. 

Thank you, Mr. Chairman. 

(The prepared statement of Dr. Gregory R. Anrig follows:) 



It."} 



161 



Te8tifiK>ny for the Subcommtttee on 
Elementary. Secondary and Vocational Education 
Committee cm Educatfcm and Labor 
U. S* House of Representatives 
March 14, 1991 

Gregc^ R. Anrig 
Pre^dent 
Educational Testing Service 
Princeton, J. 



MaHonal Test: 'Way* ■ nationwide Assessment Svstem: "Yea* 

There is the t>eg{nning of a p<^ttive shift away from what has been a 
bandwagon in favor of a national test to improve American education. I 
support such a shift Today I want to ot^ine why a national test will not 
achieve its intended purpose, and then descril>e actions that I t>elieve 
would improve teaming for all children. 

It may seem strange that the head of the country's iar^t test 
devefopment organization is opposed to a national test One of the 
central missions for which Educational Testing Service (ETS) is chartered 
as a nonprofit educational organization, however, is to promote public 
understanding of tests and of proper test use. I take that responsibility 
seriously. 

Why Not a National Test? 

Embracing the idea oi a national test to achieve educational 
excellence is like picking a referee t^ore you know what game you are 
ptaying. We first need to define what we expect students to know and be 
able to do. The National Educatton Goals announced by President Bush 
and the nation's governors in %9W was a good first step. But what does 
a goal for U. S. students to t>e first in the world In mathematics and 
science achievement by the year 2000 mean to a fourth grader in 
Michigan or Pennsylvania and to his or her teacher? What do we expect 
them to know and tie able to do in order to achieve this goal? These 
^als need to be translated into terms that have meaning for instruction, 
by subject and level. 

Such terms should not be defined by a test Any test, by its very 
nature measures only a sample of knowledge and skills. If tests are used 
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to determine the curriculum, you will end up with a sample curriculum. 
Effective instruction focuses on what steidents shmjid Icnow and be able 
to do, not on what is on a test. Effective achievement tests are valid and 
reliable measures that focus on what is taught* Rrst decide on your 
expectations and standards for student achievement. Then decide how 
best to measure this achievement 

Simply imposing a national test on schools to Jump start reform will 
aggravate even further the general feeling of teachers that education 
reform is something t>elng done tS them and their students rather than l^r 
them. Private enterprise has learned the hard way that competitive 
productivity requires involving the workforce rather than dictating to it. 
Better student achievement must be developed in the classroom and only 
the teacher can do it. Teachers should have a central role in reform if we 
are to benefit from their dedication and commitment. To date, this hasn't 
happened; a national test would represent another step in the wrong 
direction. 

No Shortcuts to ExceltenM 

Obviously, I am not against tests* I sm against using tests in ways 
that wont work and are educationally unsound. The country has had 
considerable experience with imposed testing, yet our educational 
problems persist* A recent report funded by the Ford Foundation 
estimated that 20 million school days and 100 million tax dollars (for 
direct expenses) are invested annually in public school testing. 

States and school distrtcte already have tried the testing route to 
reform. By the end of the 1970$, 38 stetes had stetewide testing 
programs; by the end of the 1980s, this number had increased to 47. 
While there have t>een some modest gains in basic skills achievement 
since the 1970s, the National Assessment of Educational Progress reports 
only mediocre performance at the middle and higher levels of 
achievement In reading, mathematics, science, writing, and civics. Other 
national indicators of student achievement indicate ihe same plateauing, 
especially at upper levels of proficiency. Despite earnest efforte and 
enormous investment of tax dollars in stete and local testing programs, 
student performance stilt falls far short of what is needed. 
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Eight years after the release of A Nation at Risk, few 12th grade 
students can demonstrate high levels of performance In reading, 
mathematics, and science. Few reach a broad understanding of 
American Institutions of government and are able to Interpret U. S. 
historical Information and Ideas. Few display an ability to produce 
elaborated responses to various writing tasks. 

The widespread public concern that led to this period of education 
reform was fueled by the belief that American children were not teaming 
welt enough for the world that awaits them as adults. The public was 
right In the 1970s; It was right In the 19808; and It sUII Is today. 

The record Is clear that we cannot test our way to educational 
exceHen<». Tests alone describe the problem; they dont solve It. A 
national test Is no shortcut to the real reform we all are seeking. 

Criteria for an Effective Nationwide Assessment System 

If a national test wont work, what will? t-et me offer nine criteria for 
an approach that I believe would work: 

• Expand the concept of educational accountability to Include 
states and communities, as well as school districts and schools. 
Accountability cannot Just rest witii educational Institutions; H must be 
shared by thMe responsible for their governance and funding. 

■ Establish standards for all subjects at Important educational 
transition points, based on a broad4}ased «>n8en8U8, tttat Include skills 
as well as knowledge, and address omployment-relevanl as wen as 
academic outcomes for all students. We dont Just need higher 
standards, we need new standards equal to the demands of the complex, 
competitive, technological world environment »»at faces today's students. 

■ Define what is to to teamed before deciding how to assess tills 
learning. Focus primarily on outcomes so that teachers have opportunity 
for flexibility and creativity In how best to achteve these outcomes. 
Provide for tevels of pertermance wHhln standards bo that progress can 
be Identified and ancouras^, not Just siHxess or failure in rerohing a 
single, fixed point 
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■ Assure ttiat students are taught what the standards call for, that 
teachers know what teaming outcomes are expected of them by subject 
and grade levels and that schools know ¥vhat goals ttiey should strive to 
achieve, involve teachers significantiyt along with otfiera, in determining 
titese standards, outcomes and ^^Is, and how best to assess them. 

■ Unk tocai creativity in assessment wi^ state and national 
indicators of pefformance. IMost assessment should continue to t>e 
devised locatly tor insfructionai purposes. Each level of government 
should stimulate assessment ideas and training In assessment for the 
educational system. Promote the use of a variety of assessment 
techniques ttiat include student-produced results based on group as well 
as individual effort Since stendards can never cover everything we wish 
children to team, encourage diversity of learning, cultural understanding 
and enrichment within ttie context of common standards. 

m Vigorously guard against creating a new system to sort and 
label children, especially those bom into poverty (24 percent of those 
under age one in 1989, according to the Bureau of the Census) who start 
childhood with educational disadvantages not of their own making. 

■ Gather and report individually-identifiable student scores only 
when that information wilt foe used to provide improved educational 
opportunity and/or additional instruction to these stodente. All other 
assessment should be conducted on a sampling basis to guard against 
test misuse. 

■ Use assessment positively as a basis for substantial incentives 
to schools and schTOt districts for progress towards achieving outcomes 
and standards, and for substantial help to schools and schMl districts 
whose studente are not progressing satisfactorily. 

e Design the system so it is dynamic and can be improved over 
time based on experience and advances in knowledge. 

These criteria do not describe a "nattonai test*". They describe a 
nationwide assessment system that builds upon the traditions of 
American education, with ite diverse student population, rather than 
imitating the highly centralized (and federalized) systems of Europe and 
Japan. An interim report of the National Education Goats Panel, chaired 
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by Cotorddo Governor Roy Romer, recommends creating such a 
nationwide assessment system by the end of ttiis decade. The panel is 
on the right track. A national test is the wrong track for education and for 
the country. 

Prmiising Mod^ fm Success - ExMIng Elements 

Elements of such a nationwide assessment system already exist. 
They can be learned from and built upon. Let me describe some of them: 

National Council of Teach ers of Mathematics. American Association for 
tha Advancement of Science. National Rese arch Counc ira Mathematical 
Sciep^g Sdygption Poard . TTiese three organizations are MtUng new 
standards and learning outcomes that are earning the respect and 
ownership of those who must carry them out. They also are involving a 
broadly representative group in identifying pit)mising ways to assess the 
new standards and outeomes. 

State Initiatives for New Wa ys to Assess Learning . States such as 
California, Connecticut and Vermont are pioneering new forms of 
assessments based on state-developed learning outcomes. In each case, 
teachers have had a central role in the initiative. 

The National Assessment of Educational Progress and Trial Sta te 
Assessments^ The National Assessment is attracting increased attention 
and credibility as The Nation's Report Card". It has an Invaluable 
capacity for tracking performance trends over time and future progress 
towards the new National Education Goals. The National Assessment 
also is becoming a vehicle for developing measurement innovations that 
are technically and economically feasible for large %ale assessments on 
a nationwide basis. Approximately 25 percent of the Hems in tiie 1990 
assessment were performance-based and tills proportion will Increase 
significantiy in the 1992 assessment tn addition, thirty-seven states are 
participating voluntarily in the Congressionally-uuthorind Trial State 
Assessments this year. Only one of these states has dropped out of the 
Trial State Assessments for 1992 and four new states have signed up to 
replace H. In June of this year, the first assessment reports wiU be issued 
that wiil permit valid state-by-state comparisons of student performance in 
mathematics. 
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Bgsnick/Tiicker Proposal for a Nationat Examlna tton System. Unlike 
propc^ls fo>* a nationat test, Resnick and Tucker propo^ a national 
examinatiOii system that is syUabus4>ased, voiuntaryt and draws upon a 
variety of assessment techniques (examinations, portfolios, projects) that 
will protect ftexibUKy and creativity for teachers at ttie local level. 

The C ollege B o ard's A dv anced Placement Program , Operational 
nationwide for 35 years, the Advanced Placement Program provides 29 
examinations in 15 subject matter fields. More ttian 330,0^ high school 
students took Advanced Placement examinations in 1989-90, an increase 
of 175 percent since 1980. These respected examinations are syllabus* 
driven, are largely performance-based (essays, portfolios, problems), and 
are nationally judged by carefully trafned assessors who are high school 
teachers and college professors. A total of 9,292 high schools across the 
country provide Advanced Placement coui^es and 1800 colleges and 
universities use Advancement Placement grades to determine course 
placement and credit. The numt>er of urban high schools and minority 
students participating successfully in the Advanced Placement Program 
has increased dramatically in the 1980s. 

National Assessment Governing Bo ard Stjindards . he Governing Board 
has begun the complex task of developing achievement levels for 
subjects and grades assessed by NAEP. While I have expressed 
concerns to the Board about the procedures used for its initial effort, I 
support the development of standards at several proficiency levels for the 
National Assessment, and the Governing Board's role in doing so. 

The Colleg e Board's Educational EQuaiity Project. The College Board 
involved literally thousands of educators and ethers in developing its 
''Green Book", Academic Preparation for College, and the follow-up 
"Rainbow Books" that describe high standards of expected 
accomplishment in each academic area for college*bound students. The 
process followed by the College Board demonstrated one viable model for 
devetoping a broad-based consensus on teaming outcomes. 

Towards A Common Curriculum and In^ructicMiaUy Useful Assessment - 
Two Remaining Issues 

t advocate achieving a consensus on standards and learning 
outcomes before deciding on how best to assess student performance. 
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Wouldn't this result in a "nationa! curriculum" feared by so many 
educators and others over the years? 

I always have felt the issue of a national curriculum was something 
of a red herring. If one were to take a heitcopter Journey across tiie 
country, touching down every 150 miles, and walk to ttie nearest school, I 
believe you would find more curriculum similarity than difference among 
schools nationwide. This Is largely due to textbooks marketed on a 
national basis by a relatively few commercial publishers. The problem 
with this is that a common curriculum is being defined de facto by 
vagaries of the textt>ook marke^lace. 

I am coming to the p^^nal opinion that educators notHi to reassert 
responsibility, as has the mathematics profession, and organize with 
others a deliberate effort to determine a common curriculum for public 
educatton in the United States. Such a common curriculum should center 
on what should be learned, giving teachers and schools flexibility to be 
creative in how to achieve this learning. American Federation of Teachers 
President Albert Shanker proposed the need to develop such a common 
curriculum In his February 24, 1991 weekly column in the New York 
TiBQga. I agree with him. 

An area in which my ideas are stMi terming has to do with what is 
called instructional assessment. Traditionally, stendardized testing in the 
United Stetes has been expected to serve two puriK^s - public 
accountability and instructional improvement While these two purposes 
are not totelly incompatible, I am tecoming Increasingly convinced ttiat 
the use of one instrument for N)th purposes may result in neither purpose 
t>eing served as effectively as It should. The El^ BoarU of Trustees 
annually publishes (as part of the ETS Annual Re(H>rt) an Accountability 
Report on matters of public concern. Ite '\990 Accountebility Report is 
titled Instructional and Accountebility Testing in American Education: 
Different Purposes, Different Needs". I append this report to my 
testimony because of its relevance to the issue of a national test. 

t thank the Sut>committM on Elementary, Secondary and Vocational 
Education for the opportunity to present my views on this important 
sub|ect 
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Chairman Kildee. Thank you. Dr. Anrig. 
Dr. Seweil. 

Dr. Sewell. Mr. Chairman and members of the subcommittee, 1 
am the acting dean of the College of Education at Temple Universi- 
ty and a fellow of the American Psychological Association. I wish 
to thank the subcommittee for inviting me to t^tify today on the 
important issue of national testing. 

As Americans, we are justifiably proud of the favorable condi- 
tions under which the majority of our children are reared. We 
know of the physical » social, and nsycholc^cal care provided to 
those born of advantaged status, ana we know of the quality of the 
intellectual stimulation associated with middle class child rearing 
practices. We know of the emotional investment of parents in the 
transmission of the culturally important elements of the society. 
We know, too, of the care with which children are expired to a 
wide variety of experiences. Those of us who are educators, 
through our research and our experience in the schools, also know 
how all of the^ factors can pc^itively affect a child's overall func- 
tioning, including the aMlity to take standardized tests. 

But there is another social reality into which a large percentage 
of America s children are born. It is a world in which their lives 
are battered by poverty, social isolation, and often racial and 
ethnic inequities. It is an environment where crime, drug addition, 
pollution, and other factors are daily realities. There are rural and 
urban communities where children attend school physically and 
psycholc^cally unprepared to learn. There are social conditions in 
which the nutritional status and health care needs of the children 
adversely affect academic achievement. It is a grim world, far re- 
moved from the environment of America's privileged middle and 
upper class vouth. It is a world in which taking and doing well on a 
standardized test is often irrelevant and sometimes imFK)ssible. It is 
my awaren^ of these conditions and of titeir impact on our chil- 
dren that makes me irrevocably oppc^^ to a i k '.ional standardized 
high school examination. 

Although poverty and a socially disadvantaged background are 
not insurmountable barriers to outstanding performance on 
achievement tests, children who experience these social ills consist- 
ently manifest lower performance on a variety of standardized tests 
when compared to children of upper socioeconomic status. There 
are communities in which lack of opportunities and low school and 
community expectations are often cited as reasons for an approxi- 
mate school dropout rate of 50 percent. 

Hie Federal Government is fully aware of th^e situations. The 
State govenunents have reported the&e sutJstics based on their 
own testing pn^rams. The local communities can identify each and 
every school in which substandard academic performance has 
become a political footbalL The local and State governments can 
also provide evidence that the expenditure per pupil in these low 
performance schools is usually between one-tiura to one-half of 
that of the wealthier communities. The issue, ^erefore, is not a 
lack of awareness of the difficulties we face as a nation but, rather* 
th» tendency to propose simple solutions for complex pn^lems. A 
national graduation exam is this type of a simple solution. To 
quote from the 1990 report of the National Commission on Testing 
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and Public Policy, **We cannot test our way out of educational 
problems." 

But despite the widely recognized problems with standardized 
tests, there is still a call for a national test. While there are many 
reasons cited in favor of a national test, I would like to focus on a 
few of the major ones and give my reactions to them. 

Point one: We need to know what works. This argument usually 
focuses on the issue of accountability, of finding out which of our 
schools is doing well and which ones are doing poorly. It is often 
raised in the context of how tax dollars are spent and what com- 
munities are getting for these dollars. 

While I recognize why this argument is raised, I have several ob- 
jections. First of all, a n'^tional test cannot tell us anything more 
than we already know. We already know that poor, urban schools 
have lower mean scores on standardized tests than middle class, 
suburban schools. Merely proving this one more time does nothing 
to remedy the problems that have given rise to this discrepancy- 

Secondly, America is already spending over $700 million per year 
on toting. Although there is no national test, the emphasis on test- 
ing at the expense of instructional time has already exceeded rea- 
sonable limits. According to a 1985 national survey. 47 of the 50 
States mandate some form of statewide testing on student attain- 
ment. In my opinion, one of the major lessons we have learned 
from all this testing is that implementing testing programs has no 
discernible impact on student achievement. There is no reason to 
believe that a national test will have any different outcome. 

Point two: A national test can provide valuable information for 
employers, colleges, and other consumers of our educational 
output. This is another version of the "what works" rationale, 
except that the focus is on providing a way to differentiate between 
students at the time they graduate from high school. 

I have several concerns about this argument My primarv objec- 
tion is that this argument runs counter to an essential American 
value, that change and growth are an iiiherent part of life and that 
redemption is always passible. We all know people who, as they 
grow older, are tetter able to cof» with their difficulties. Some are 
able to leave a dysfunctional or poor family through hard work in 
diverse jobs. Othere find that they begin to learn how to handle 
family expectations, overcome learning problems, and in time de- 
velop a better sense of worth. What will become of these people if 
they are effectively screened out of greater possibilities at the time 
they graduate from high school? 

In addition, if we accept the need for some type of accountability 
at the time of graduation from high school, there is no implied re* 
lationship between this accountability and testing for instructional 
purposes. If excellence in academic achievement is the central goal 
of a national test, how will the scores enhance achievement and in- 
struction? I think there is ample evidence to indicate that stand- 
ardized tests do not achieve this objective, as the experience in my 
own State of Pennsylvania indicates. 

The State of Pennsylvania mandated a test to be given to all stu- 
dents in the State to assess the performance levels of schools. Mon- 
etary help was provided to support remedial efforts where indicat- 
ed. Schools in the most aiTIuent segments of the State were com- 
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pared to those in the most economically depressed areas in which 
low performance was already documented. The promised resource 
for remedial help was rather quickly discontinued, yet the compari- 
son persisted. If this example can be generalized to the national 
level, why would we spend an additional $100 mUlion for testing 
when the political reality of the local community with the constitu- 
tional control over the curriculum and school financial support is 
unlikely to change? , . » a.- r 

Point number three: Almost all of the industrialized nations of 
the world have such an examination. This argument compares the 
United States with countries such as England, Germany, and 
Japan, and seems to imply that th^re is some connection between 
the economic advantages that these countries are experiencing and 
the existence of a national exam. 

While it is true that many countries have a national exam, to 
compare and contrast divert culture is simplistic and misleading. 
From a purely pragmatic perspective, the situation in the united 
States and countries like England and Germany and Japan are not 
the same. These countries test far fewer students than we do, and 
they test far more selectively. This allows these countries to use 
exams which demand integration and synthesis. The possibility of 
reproducing this type of test at a national level m this country 

seems remote. „ , , , , * * *u ■ 

More importantly, countries like England and Japan test their 
citizens to limit access to higher education and to discriminate 
among those who will have access to better schools. It is unlikely 
that any of the proponents of a national test woulu make this one 
of its explicit goals. However, I believe that the proposed national 
test, rooted in the assumption of higher national standards, might 
simply be a thinly veiled effort to use a scientific rationale to ad- 
vance an elitist sMial structure in the United States. If not so, why 
then, given the rich American tradition of local control over the 
educational enterprise, should a national high school test dictate 
employment, a high school diploma, or college admission? 

Point number four: A national test will have the desired outcome 
of showing what works without the negative outcome of leading to 
a mandated national curriculum. It is difficult to take this argu- 
ment seriously. Proponents of the national test seem to forget that 
curriculum and achievement tests go hand in hand. In fact, if they 
don't, what is the st measuring? Moreover, these proponents also 
argue that the existence of a national test will not lead to teaching 
for the test. Again, it is difficult to accept this argument. Already, 
without a national test, in a recent Gallup Poll, 73 percent of the 
teachers said that they feel pressured to prepare children for test- 
ing; 57 percent said that the testing determine what they teught. 
Let us be clear about this issue: A national test will inevitably lead 
to a n-^tional curriculum. Even more importantly, invaluable class- 
room time will be spent preparing for the test. 

I would like to summarize my presentation bv pointing to the 
fact that testing has always been a mechanism subject to the pohti- 
cal realities of the community. There is a seductive argument that 
attributes to testing the salutary value of improving the opportuni- 
ties of low income and minority children by reducing the pervasive 
class and ethnic bias associated with personal, subjective judg- 
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ments. The proponents of standardized tests may also believe that 
the social concept of equality of educational opportunities is best 
served by the objectivity of tests. 

My own belief, based on research and years of experience in the 
schools, is that differential socioeconomic circumstances make 
standardized tests inherently biased gainst poor and/or minority 
children and, consequently, negatively affect educational opportii^ 
nities and success. 

The advocates would lead us to believe that American education- 
al excellence will lead to the promised land and that one of the ve- 
hicles to achieve this objective is to develop a national standardized 
t^t. Even if this argument is true — and I hope that I have demon- 
strated why I don't believe that it is— the question could still be 
asked: For whom is this supposedly excellent system designed? For 
many it will not be a promised land but a land of shattered 
dreams. 

Thank you, Mr Chairman. 

[The prepared statement of Dr. Trevor Sewel? ibllows:] 
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Mr, Chairman and "Berbers of the 5>utu'ommi t t^>e : 

I am Trevor St^wf*!!. I am Acting Hean .>f 'ihe Tolle'^t^ of 
Educt^tlon at Tt^mple Ifniversity and »3 ] J jw of tnp Amt^ri .^n 
Psychological Association. I wish to thjnK the Subc^>Ti'ni t toe fnr 
invitinq me to testify today on th»^ isiJjf- n<»ti'>njl test in?. 



Introduction 

As ATrer 1 ;'cins, we .ir*-- j t.*s t i f i .-J h 1 y pro'jd of the fi'T-n^ie 
'•onditions anMtT ^fhich the in.ijority of our chiii^ren irr* r+^^-ed, 
We know >f the physical, social and psycho 1 o<u I Crsre provi.'ed 
to those horn >i .Tdv.in^ iied status, ^nd we know ot the quality . f 
the 1 nt-^ 1 lef*t o.^ 1 sr;nuU3tion rissori^ted with m idd le-r 1 ass child- 
rear in ? pr ,ict 1 '"^rv . we know of the emotional invests t of 
parents in the t r.m'^T^ i ss i on of culturally important elemv:. of 
the society. We knc'w, too, of the care with which chiidten v^re 
exposed to J wide variety of experiences. Those of us who .?re 
educitors, throoih o-jr research and our expf»rience in the 
schools, ^Isc know how all of the«;e factors can positively affe-^t 
a chjid*s ov«:Tan functioninq, includii*r| the ability to t.ike 
standardized tests. 

But therp IS another social reality int> which a Itr^*^ 
percent a'3e of America's children ^re bocn. it is a wot I 5 in 
which their lives are battered by poverty, sonal isn Nation ind 
often rarial and ethnic inequities. It is an environment where 
crimn, dru'j addiction, pollution and other far^tots are daily 
realities. There are rural and urban comir "Cities wht»re children 
attend school physically and psycholoqically unprepared to learn. 
There are so-i/il ,-onditions in which the nut r 1 1 1 (.>na 1 5^tatus -^nd 
health care needs of the children *id tersely affect a cad em i-: 
-achievement . It is a qrim world far tenioved from the env i r on'ni-n t 
of Americans privileged middle and upper class youth. Tt is a 
world in which taking and doxn^ wall on a standardized test is 
often irrelevant and sometimes impossible. It i s my awareness of 
these conditions and of their impact on our children th^t makes 
me irrevocably iJppo?;t»d t <■* national standardized huh no-* 
exammat ion . 

Aithouqh poverty and 3 socially 6 i sad\/^nt t/ack«i round are 

n<>t insurmountable barriers to oat stand in f>e r f o r ma .nee r; n 
achievement tests, children who experience these social ills 
consistently manifest lower performance on a variety of 
standardized tests when compared to children of hnher 
socioeconomic st-itus. There are communities in whi^h lack of 
opportunities and low srhool and community expectations are often 
cited as the reasons for an approximate school drop-out rate of 
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p*»ccent. The Federal government is fully aware of these 
situations. The st-^te novernments have reported thost? statistics 
based on their own testing pcoqtams. The local communities can 
identify each and every school in which substandard acadetjic 
performance has become a political football. The local and State 
qovernments '^an also provide evidt^nce that the expenditure per 
pupil in these low performance schools is usually between 1/3 tr» 
1/2 of that of the wealthier communities* The issue, therefore, 
is not a lack of awareness of the difficulties we face as a 
nation, but rather the tendency to propose simple solutions for 
complex probi«Tis. A national graduation examination is this type 
of simple solution. To quote from the 1990 report of the 
National Commission on Testing and Public Policy: "we cannot test 
our way out of educational problems". 

Argument a Against a National Yest 

Put despite the widely recognized problems With standardized 
tests, there ;s still a ca I i for ^ national tes . While there 
ace many reasons cited m favor of the nationa. test, I would 
like to focus on a few ma^or ones « i give my reactions to them, 

il) We need to know what uozka* This argument usually focuses 
on the issue of accountability; of finding out which of our 
schools IS doing well, and which ones are doing poorly. It is 
often raised in the context of how tax dollars are spent ^nd what 
communities are gettimj for theso tax dollars. 

w ile I recognize why this argument is raised, I have 
several objections. First of ail, a national test cannot tell us 
anything more than we already know. We already know that poor, 
urban schools have lower mean scores on standardized tests than 
middle-class, suburban schools. Merely proving this one more 
time does nothing to remediate the problems that have given rise 
to this discrepancy. Secondly, America is already spending ov^r 
70» million dollars >er year on testing. Although there is no 
national test, the emphasis on testing at the expense of 
instructional time has clearly exceeded reasonable limits. 
According to a i98S national survey, 47 of the S3 states mandate 
some form of state-wide testing of student attainment. In my 
opinion, one of the major lessons we have learned from all of 
this testing is that implementing testing programs has no 
discernible impact on student achievement. There is no reason to 
believe that a national test will have any different outcome, 

<2) A National test can provide valuable inforaatioo foe 
eaployerSf colleges and other conauwers of our educational 
output. This is another version of the "what works" rationale, 
except that the focus is on providing a way to differentiate 
between students at the time they graduate from high school. 

I have several concerns abot:t this argument. My pri*^ary 
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objection is that this argument runs counter to an esjsential 
American value ~ that chanqe and qro^^th are an inherent part of 
life, and that redemption is always possible. We all know people 
who, as they get older, are better able to cope with their 
difficulties. Some are able to leave a dysfunctional oc poor 
family through hard work in diverse jobs. Others find that they 
begin to learn how to handle family expectations, overcome 
learning problems and in time develop a b*?tter sense of self- 
worth. What will become of these people jf they are effectively 
screened out of greater possibilities at the time they graduate 
from high school? 

In the past, the special feature of our American educational 
system has been its emphasis on access to lifelong learning. For 
example, an older :>erson can return to high school even after 
dropping out; a community college can serve as a gateway to 
opportunity; state colleges and universities have provided 
excellent higher educations at bargain rates; doors have opened 
for "late bloomers" when they were ready to learn; immigrant 
children have been able to obtain a college degree. in short, 
America has continued to be the land of educational opportunity. 
Will a standardized examination continue this tradition, or will 
it screen out from further education people from diverse 
cultures, from different races, of different genders, and ot 
varied ages? 

In addition, even if we accept the need for some type of 
accountability at the time of graduation from high school, there 
is no implied relationship between this accountability and 
testing for instructional purposes. if excellence in academic 
achievement is the central goal of a national tei^t, how will the 
scores enhance instruction and achievement? t think there is 
ample evidence to indicate that st<*ndardi r<>.i tests do not achieve 
this objective, as the experience m my own state of ^'ennsy 1 van i a 
I nd I cates. 

Thi^ State ol Pennsylvania mandated a test to be given to <il 1 
students in the State to assess the performance levels of its 
schools. Monetary help was provided to support remedial efforts 
where indicated. Schools in the most affluent segments of the 
state were compared to those in the most economically depressed 
•ireas in which low performance was already documented. The 
promised resource for remedial help was rather quickly 
discontinued, yet the comparison persisted. if this example can 
be generalized to the national level, why would we spend an 
addit)onal $100 million for testing when the political reality of 
the local community with the constitutional contrnl over the 
curriculum and school financial support is unlikely to change? 

(3) Almost all of the industrialized nations in the world have 
such an esamindtion. This argument comparer the United states to 
countries such as Kngland, Germany and Japan, and seems to imply 
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that these enuntrie.s <? xpf r lern^ ^ the existed *t- of 

nat I ona I exv^m . 

W h i I It IS t r u t"* t n ^ t Ti a n y o r? u n r r i ♦» s h .1 v f > j n t i o n a 1 
OS a'T^ 1 n 1 1 I ^^n , to rofv^pjr^^ .?nd c<intr.ist diver so cultures is 
fj ifTpI t *^t jnd f;l I Hvi » From a purely praqmatic porsp<-ct ive, 
the sitiiits>n in the '),.S. and countries I i kf» Knil.md, Gi>rmany rind 
Japan ^ro f^ot the s.im^*. Tht»s<» countries test far fewer students 
th^n we do, ind they test far more selectively. This allows 
th*>s4> .-countries to ur. o e5%say examinations which demand 
mteiration and synthesii^. The possibility of rpproducin<l this 
type ot test at a national level in this country sewns ri»mote. 

*^^-)ro impar t ant I y ^ an tries I 1 Ke lin^iland and Jap^m tost 
tni^ir citizens to limit access to hi-^her education ;ind to 
ii scr i:ninate amon t those who will have access to better schools. 
It IS unliKeiy that any of the proponents of a national test 
would make this one of its explicit qoals. However, I believe 
that th> proposed national test, rooted in the assumption ot 
hiqher national standarris, miqht simply he a thinly veiled effort 
to oso a scientific rationale to advance an elitist social 
structure in the United States, If not so, why then, ^iven the 
rich American tradition of local control over the educational 
enterprise, sh<Mjld a national hiiih school test dictate 
emplt^yment, a hiih school diploma or college admission? 

{4^ A national test will have the desired outcoae of 8howin<| 
**vhat wo?ks* trithout the negative outcoise of leading to a 
Bondated national carriculum* It is difficult to take this 
arqoment seriously. Proponents of the national test seeii to 
ft rt|et that curriculum and achifrvement tests qo hand m hand. In 
fact, if they don't, what is the test •neasurin^? Moreover , 
these proponents also argue tnat the existence of a national test 
will not lead to teaching for the test, Arjain, it is difficult 
to accept this ar^iument. Already, without a national test, in a 
recent qallop poll, 7 3% of the teachers said that they feci 
pressured to prepare children for testing; 57% said that tests 
determine what they tauqht* Let us be clear about this issue: a 
national test will inevitably lead to a national curriculure. Kven 
more importantly, invaluable classroom time will be spent in 
preparing students for the test. 

Even if we accept the possibility that a national high 
school test can be developed to me*»t acceptable technical 
properties of validity and reliability, what will be the social 
consequences? This question is of particular importance in liqht 
of the absence of a mandatory national curriculum and textbooks, 
the discrepancy between local communities* per pupil 
expenditures, and the adverse impact of social-class and racial 
inequities deeply entrenched in American education- 
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I would like to summer i2t> my pre son tit ion by pointing to the 
fact that testing has always been a mechanism sub}ect<»d to th*? 
political realities of the community. There is a sedurtwo 
arquraent that attributes to testing tht- salutary value of 
improvinq the opportunities of low income and minority children 
by reducirtq the pervasive class and ethnic biases associated witn 
personal sut>}ective Judgements. Th.- proponents of standardized 
tests may also believe that the social concept of equality of 
educational oppottunities is best served by the OD}ecti\;ity of 
tests. My own belief, based on research and years of experience 
in the schools, is that differential socio-economic circumstances 
make standardized tests inherently biased aqamst poor and/or 
minority children, and consequently negatively affect educational 
opoor ton 1 1 ICS and success. 

The advocates would lead us to believe that American 
educational excellence wi I ] lead to the promised land, and that 
one ot the vehicles to achieve this objective is tr> d«»velop a 
national standardized test. Kven if this argument is true, and I 
hope that T have demonstrated why I don't believe that \t is, the 
question could still be asked: for whom if? this suppose.'»ly 
excellent system desiqned? For many, it will not be a promised 
land, hot a 1 .md of shattered dreams. 
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Chairman Kilder Thank you very much, Dr, Sewell 

The subcommittee has heard some concurring and some contrast- 
ing views on the question of as^ssment and testing. I would like to 
ask a ^neral question and have any or all of you respond to that. 

Testing or as^ssment, for the most part, takes place at the end 
of an experience or the end of a process. What do we discover in 
tCTting that can help us to improve that experience or process? As I 
mentioned yesterday, testing is generally an om^ point. What do 
we do with alpha, beta, gamma, delta? How do we make testing 
helpful in the improvement of education? 

Governor. 

Mr RoMER. I think that a test ought to do four thmgs. First, it 
ought to organize one's work; it ought to motivate them; secondly, 
it ought to accommodate individual differences in learning; third, 
there ought to be real consequences for the learner out of a test; 
and, fourth, you have got to have clear standards that you are 
reaching for. Let me go back to the pilot test again, because it is a 
veiy good illustration. 

When you are teaching somebody, you teach to the test, and if 
you have got the right t«?t, it is a good thing. If you have got the 
right pilot test, that test, as you go through that course of training, 
can tell you, you don't need to spend your time on this, you have it, 
but you do need to spend your time on that. 

S«rondly, there are real consequences to that test which reward 
someone for having done it right, and this is one of the things that 
I just think, as I listen to all of us this morning, even though I was 
not advocating a national test but a national system, I would not 
like us to conclude that the system as we have it is okay, because 
we hf^ve the level of educational performance in this Nation which 
is quite inadequate because people think it is adequate; that is the 
problem. We, frankly, out there think, "Hey, as a nation, we are in 
pretty bad shape, hut our school is doing a very good job," and this 
wrong, and we need to have assessment systems that make us, as 
parents and as students, know that we are ill educationally, as Sen- 
ator Brock has said, compared to what our vocational requirements 
are going to be when we are adult. 

Chairman Kildee. Senator. 

Mr Brock. I will try to respond in a slightly different direc- 
tion—I agree with that— but what you can do with tests. I don't 
think I have heard anylx^dy suggest a national graduation test; 
that isn*t even an i«sue before us, as far as I know. What I thought 
we were talking about was a series of tests and evaluations and as- 
sessments ^that allow a student to know where he is in the spec- 
trum of progress and allow us to know what we need to do get 
them up to the full scale of productivity that they are capable of. 

So v/hat a test can do for you— a series of tests or a series of as- 
sessments, whatever word you want to use— is to say that by the 
fourth Krade you have gotten to this point and you are not going to 
make it to the level that you want to achieve by the time you are 
16 or 17 unless you change your level of performance in this par- 
ticular area. 

So it allows the student to be given the knowledge of what they 
have got to do to begin to move in the right direction. It allows the 
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parent to know and allows the teacher to know where to focus 
their efforts. 

Canada has a wonderful program that was developed by private 
enterprise, funded by government, called Yes, Canada. You can sit 
down at a computer, and in 30 minutes that computer will analyze 
you. It doesn't matter whether you are three or 80, it will start 
talking to you in your language, and, from then on, if you say, "I 
want to study, off the mainframe data base, Spanish," or English, 
or math, today, the computer works with you for that hour of in- 
struction at that level, and the teacher is sitting in the room with a 
master bc^rd, and the teacher will all of a sudden see a flashing 
light, and the computer will say, *Teacher, Bill is having trouble 
with his algebra; go over and work with him as an individual/' 
Then you are beginning to use the teacher's real talents on an indi- 
vidual basis. We don't do that today. We lecture, we hector, and we 
are terrified of evaluation and ass^ment. 

You can use those assessments, I think, to give the students the 
tools they have got to have to make that progress and to give the 
teacher what they simply have to have. We are not giving our 
teachers any clear message tcwday. They know what it takes to send 
a child to college. They have no idea whatsoever what it takes to 
prepare a child for work, and 70 percent of these kids are not going 
to finish collie. 

Chairn^f^n Kildee. Dr. Sewell. 

lir. SewelLh I think what I have heard, sir, is a distinction be- 
tween the national test and a diagnostic test. I think what can con- 
tribute to the educational standard is some form nf di^nostic test. 
That cannot accomplished by a national test. A national test, by 
itself, would have to incorporate items that are more generalized to 
different socioeconomic groups and different school experiences 
across the country, and the items will be so selective that that 
could not have that diagnostic flavor to achieve that objective, and 
the information that we need to know as to what to do to promote 
excellence in those classrooms is already known in most of these 
schools. 

Chairman Kildee. Yes. 

Dr. Anrjg. Mr. Chairman, let me raise a concern in response 
particularly to the poinvS raised by Secretary Brock. Involved with 
part of what he was saying is a concern that I have with this issue 
nationwide, and that is that, in the long run, we turn around and 
put all of the responsibility back on the student for whether he or 
she has made the mark. 

I think we need to broaden our concept of accountability so that 
when someone is not succeeding, we take a look not just at the stu- 
dent but the teacher, the school, the community, and what that 
community has provided to the schools, and the State for its re- 
sponsibility for that school. We tend to end up, when all is said and 
done, saymg the student made it or didn't make it, and I think 
there ought to be greater accountability than that. 

Part of the problem reflected in your qu^tion, Mr, Chairman, is, 
we tend in the United States to try and use one instrument to do 
two purposes: improve instruction and take care of accountability. 
ETS, for instance, is working in areas of instructional assessment. 
We are working in the Pittsburgh public ^hools on portfolio as- 
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s^ments in art, music, creative writing, doing some very interest- 
ing stuff, which is great for the classrrora, and great for the teach- 
er, and helps them improve instruction, but, as a former commis- 
sioner of education in Massachusetts* I would have a tough time 
going tefore the Joint Education Committee of the State l^isla- 
ture and reporting that we have great portfolic^, because they just 
don't want to hear that. What they want to hear is can the kids 
read and at what level, and they want a fairly simple answer to 
^hat. 

So I think we need to look at ways of accountobility measure- 
ment and ways of instructional measurement, and I suspect that 
we are not going to come up with one instrument to do both of 
those. 

Chairman Kildke, I want to go on to the next question, but I 
mentioned this yesterday, too, to the other panel I teught Latin for 
10 years, and I can recall one time trying to get across the concept 
of sequence of tenses, when you go from the indicative to the sub- 
junctive mood, and finding out ^iiat I really wasn't doing a very 
good job. I understood it, because I had spoken Latin for four years 
in college, but I couldn't figure out why they couldn't understend 
it. Finally, through testing, I found out, obviously, they did not un- 
derstand sequence of tensra. So I went back to the drawing board 
and put together a presentation that worked. Even the slowest stu- 
dent was able to master the idea of sequence of tenses. 

Can testing help redesign methodolc^? Can it help math and 
science teachers collectively or individually? How can we take that 
back and really improve the quality of teaching? 

ttoes anyone want to respond to that? 

Bill. 

Mr. Brock. Of course, the test can do that. We have learned a lot 
about testing, and, with rospect to Dr. Anrig here, I think that is 
what he was saying. I think that is precisely the purpose of an 
evaluative system. I like the phrase ''assessment system,'* t^ause 
that is the proper description of what we are talking about, and it 
clearly gives the teacher the tools that they have got to have to 
find out why there is a problem, define theproblem, and then pro- 
vide the proper response to that problem. That is what a test does 
if it is really done well. 

Chairman Kilde^s. Governor. 

Mr. RoMER. Mr, Chairman, just take the fourth grade on math. I 
think one of the most important things we need to get together on 
is what are the levels of achievement and knowledge that we need 
to shoot for in the fourth grade, and then the test ought to clearly 
measure to what degree we are accomplishing that. 

One of the things that seems so obvious to me is that all of us 
can learn but we learn at different rates. It takes some of us lo iger 
to learn than others. Again, I don't want to overuse the pilot exam- 
ination, but to get a private license some people take 36 hours, 
some 46 hours, but we may have equally good pilote: it just took 
somebody longer to learn. 

If we had to test in the fourth grade, for example, on math and 
we found out that three-fourths of the class are ready now to pro- 
ceed to the fifth grade, and we had this 60 days before the end of 
the school term, but if that b^^hool were simply organized so that 
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we could take that one-fourth of the class and say, "Look, we have 
an option of dumping you into probable failure or keeping you and 
giving you that extra effort"— that you needed to do in terms of 
your Latin students— **to accomplish this before you go on/' that is 
something that we need to incorix)rate into our system. 

What ought to be fixed is what vou need to know and h« able to 
do. What ought to be variable is the time you take to do it. Unfor- 
tunately, much of our education is organized the other way around. 
What is fixed is the number of days we sit in the seat. What is 
variable is what you know when you get out of that class and go 
into the next one. 

So testing can be diagnostic, and I think Gregory Anrig said 
something very imj)ortant in his testimony, and that is, there 
ought to be no test given unless it has an instructional conse- 
quence^ 

Chairman Kildee, Mr. Goodling. 

Mr. GoCDLiNG. I wanted to mention in my opening remarks, and 
I forgot it even though I had it on the top of the list, that General 
Schwarzkopf said a lot of interesting things during thc^ 100 days, 
but on the 101st day he said something that really caught my at- 
tention. He said it was the best educated fighting force the world 
has ever known. And then I heard him later, and he talked on the 
same subject and reminded everyone that it wasn't the high tech 
that won the war, it was that educated force that knew how to 
handle the high tech. He also reminded us that we designed, devel- 
oped, and produced far better than any of the others who were— he 
didn't sav it quite that way, but that is what he was really 
saying— than any of the other forces that were part of it. 

I want you to know that I wrote these notes before Dr. Anrig or 
Dr. Sewell testified, I didn't write them after they testified. 

The first question that I have is, who determines what every 
youngster in fourth, eighth, and twelfth grade should know and 
master? Designing the test to measure is simple; probably anyone 
could do that. 

Dr. Anrig. I wouldn t quite agree with that. sir. 

(Laughter.) 

Mr. GooDUNG. But who makes that determination? And it might 
be easier in math than it would be perhaps in social studies or lit- 
erature. So who determines that? Whoever wants to respond. 

Dr Anrig. Let me take a first crack at that. I was once described 
by a graduate advisor as stubborn like a battleship sinking with all 
guns blazing. So let me try on this one. 

I am a little concerned about some of the suggestions that have 
been made for a national board to do that. With all respect, I don*t 
know how you would be able to set up a system which would have 
widespread crf^dibility and acceptability by putting it all into one 
board. 

I think, rather than thinking about a separate board, the Con- 
gress may already have available to it a resource that could be 
broadened to do this kind of function that already has a lot of 
credibility in all of the communities that will be effected by that, 
and that is the National Academy of Sciences. It has standing com- 
mittees in various areas. It has just created one in education It has 
a very careful mechanism for selecting people, for issuing reports. 
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for involving const nsus, and I would encourage, in response to that 
quMtion, Congressman Goodling, that you consider an existing re- 
source already out there that might be a better one to turn to than 
trying to create a new board where the Congress and the President 
and others would appoint to it, a id I think a lot of i^ple would be 
nervous about that, and I would be among them. 

Mr. Brock. There is a different approach to it, and that is to say 
that if our kids have to compete with the young people cf France, 
or Japan, or Sweden, or Brazil, that we ought to see what they 
know, and that global standard could be shaped within the Ameri- 
can curriculum, within the American experience, by a responsible 
group that this Congress, as a responsible body, could appoint, It 
isn't a board that has some ultimate source of wisdom out of their 
own genius. What we have got to do is to understand that we are 
part of the world and to compete in that world we have got to have 
our kids as well prepared as theirs are in a range of subjects, not 
just math and science but in human relations skills, in functional 
skills that allow them to be pnxluctive as human being^. Those can 
be devised and derived irom the global experience. 

Mr. GooDUNG. Mr. Secretary, are you saying that our youngsters 
who have the same privities, the same l^nefits, the same opportu- 
nities, as youngsters have in France and Germany and Japan 
cannot compete with them? I don't understand. 

As you know, they have testing programs that I'mit who can go 
where and do what. In Japan, as a matter of fact, they know by the 
time the child enters kindergarten how surcessful that child is 
going to be and what they are going to be doing the rest of their 
life. 

I interview every year for the military academies, and I have to 
say that every year the young people I interview are far superior to 
the people I interviewed the year before. 

We have a totally different zystem in this country, a totally dif- 
ferent makeup, and I think where we get into trouble is when we 
try to compare our overall system with the best in every other 
country, and 1 don*t know that we can do that. 

Mr. Brock. That is simply not accurate. Congressman, with all 
respect to you. Our best only equal the average in Singapore, 
Japan, Korea, and Taiwan. 

Mr. GooDUNG. On some sf^^ific tests, Mr. Secretary. 

Mr. Brock. On v'rtually every international 

Mr. GooDUNG. Not on their ability to produce. 

Mr. Brock. Yes, Mr. Congressman. 

Mr. GooDUNG. Or we would be flat on our e right now and 
would have lost that war in 10 days rather than in it in 100. 

Mr. Brock. Let me surest to you that the best single training 
program in the world is run by the U.S. Department of Defense. 
Those kids, first, came out of high school, second, volunteered, and, 
third, went through the best training prc^am that is available to 
any military anywhere in the world. Now that is the way we ought 
to be doing it but we are not providing that. 

This great American tradition of ours that believes in equal 
treatment for all children is letting 50 percent of our kids drop out 
in most of our major urban centers. 
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Mr. GooDUNG. And testing is somehow going to bring those fami- 
lies t(^ther, make them more literate? 

Mr. Brock. If it gave them a better understanding of what they 
needed and the 

Mr. GooDUNC. Lord, they know. They know what they need. We 
have sp^nt millions and millions of dollars determining what they 
need. 

Mr. Brock. Let me tell you of one cross-cultural study between 
Seoul, Korea, and Chi(^o, Illinois. In that study, parents in both 
communities were asked how their students were doing: Are your 
kids getting a world class education in math and science? The par* 
ents in Chicago said y^; the parents in SeoaL Korea, said absolute- 
ly not. 

Mr. GooDUNG. Be vei^, "/ery careful. 

Mr. Brock. Wait a minute now. Hear me out. In Chicago, 76 per- 
cent of the parents thought they were getting a great education; in 
Seoul, Korea, 23 percen*^ of the parents were satisfied with their 
education. The fact is that the students in the schools in Seoul 
were significantly better educated than the students in Chicago. 

So our kids and our parents aren't even being told what the facts 
are. So an assessment system would help; yes, it would help. 

Mr. GooDUNG. Be very, very careful when you use Chicago, and I 
won*t steal from Charlie Hay^. 

Mr. Brock. Well, shall we try Philadelphia? 

Mr, GooDUNG. Maybe they were interviewed where the politi- 
cians are in two portions of Chicago, where most of those children 
go to private schools, where those politicians who are responsible 
for that area make very, very sure that they don't change the State 
reimbursement prc^am, because they want the rest of Chicago to 
get only maybe 13 percent while the other areas of Chicago get 
much more. So be very, very careful. 

All I am trying to point out is, it is so easy to compare apples 
and c ranges, but, boy, if ever you people on the panel should learn 
from each other; you need each other out there if ou are going to 
move anything. 

You know, we have a powerful i^rson in the White House who, 
after they set up the governors' conference with the President, I 
said, "Wonderful idea. You can provide leadership as governors 
and as President. You can't change anything. You can provide 
leadership. Now you had better bring the people in who can make 
the change." And the response was, "Then I d have to bring the 
trouble makers in." And that is a tragedy, and that is what I am 
worried about, that we are going to go off here and we are not 
going to have the trouble makers involved who are out there every 
day on the line. 

So I think we have got to be very, very careful. It is so easy to be 
simplistic. You take a Japanese society, and you look at that family 
unit, and you look at that culture. Now they will soon change; they 
are becoming so Americanized, you know, they will fall. But you 
can't compare us with that culture. 

Well, I am talking in terms of, they will be spending more time 
in the disco than they will be spending on homework, et cetera. 

Mr. Brock. Is that good? 

Mr. GooDUNG. Pardon? 
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Mr. Brock. Is that good? 
Mr. GoODUNG. Is that good? 

Mr. Brock. Yes. Is that what we want? Do we want them to emu- 
late us rather than our doing better? 

Mr. GooDUNG. Well, you know, the last time I visited, they were 
trying to make their school system more like ours. I said, That*s 
interesting. Ours used to be like yours, and we are trying to make 
ours like yours now, and you are trying to make yours like ours.'* 

But let me ask you, Mr. &cretary, will knowing more precisely 
what each child knows drive teachers to teach better, drive parents 
to parent better, drive students to try harder? Will it help us at- 
tract the brightest and best to the profession? Will it cause parents 
to work and stay together to provide the care and nurturing of the 
child, and working with the child and the school? Will it change 
teacher training institutions? If it will do all tho^? things, then I 
am all for it. 

Mr. Brock. Congressman, I have great respect for you and for 
the question you are asking; it is the right question. There is no 
silver bullet m American education. Testing will not achieve all 
three things. It will give us one of the tools that we need to achieve 
those things. If parents and teachera are given more information, 
they will act on that information. 

I think we make awfully facile arguments that we are a multi- 
cultural society. I consider that a strength, not a weakness. But 
when people say something like a t^t is to used to screen out, 
dear Lord, what are we doing now? Twenty-eight percent of our 
kids who go to high school never finish, almost one out of three. If 
they live in Chicago, or Philadelphia, or New York, or Buffalo, or 
Los Angeles, or any of 15 ether m^jor cities, 50 percent of them 
don't finish, not just 28, 50 percent. The minority kids of this coun- 
try are underserved. This rich American tradition forgets people 
when they are poor, and I don't understand. 

I had an article, Mr. Chairman, an op^ in your paper in Michi- 
gan, in which I said maybe the mc^ important thing we can do is 
to challenge the constitutionality of public education, because it is 
not serving all of our kids and giving them an adequate education, 

Testing isn't the only answer, assessment is not the only answer, 
but it is a tool, and if you don't know what you are doing, how in 
the world do you correct it? You have got to start with some knowl- 
edge base, and that is what I think we are suggesting here, not a 
test at the end of the process, a test throughout the process so that 
they can adjust themselves and learn from that, as you suggested, 
Mr. Chairman. 

Mr. GooDLiNG. Well, it was mentioned, of course, and I think Dr. 
Sewell wants to talk on that. We have done all this testing, we 
know where the weaknesses are, and, as Dr. Sewell pointed out, 
then they said they were going to send us some money to do some- 
thing about it. They didn't send us any money to do anything 
about it in the city of Philadelphia, for instance. 

Dr. Sewell. 

Dr. Sewell. I think, sir, the evidence is quite clear. We know 
how some of the schools are functioning. We know the problems of 
the schools. We know that in some of the schools a teacher-student 
ratio of 35 to one isn't going to work for those students. We know 
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that there is a lack of resources. The parents know that informa- 
tion. There has been an abundance of toting in those schools that 
have clearly pointed to the problems. I think the solution is some- 
thing other than a national test. More information will not provide 
the remedial help that is necessary. 

Mr. GooDUNG. I would close just by saying that we decided up 
here, because we heard a comment made in the testimony, and we 
decided that multiple correct answers will probably take care of 
the multiple differences of students in a national examination. 

Chairman Kildee. Governor, did you want to add ^mething 
before we go to the next questioner? 

Mr. RoMER. Well, Congressman, there ar^ schools where we obvi- 
ous! v know and there are places we know we are failing and we 
need not to solve it by more tests but by better instruction and 
more attention. But let me suggest that in Colorado we are l(^ing 
it, I think, along with 49 other States in terms of our comparative 
advantage of skill levels in the world. 

I think Senator Brock is correct, the facts are there, and the 
comparison we ought to look to is not just in Desert Storm, we 
ought to look at Toyota, which is expanding its capacity and Gener- 
al Motors which is cutting back its capacity, because that also re- 
flects skill levels, skill levels of a people. 

In terms of our work force and our skill levels and our ability to 
produce, we have lost substantial advantage in the last 20 years. 
The problem is this, in my judgment. This Nation has had it good 
since World War II. We have had it so good we think that to con- 
tinue as usual is going to continue to have it good for us, and it 
isn't going to be that way. We need to recommit and redouble our 
effort on education in this Nation. 

I think your first question was, is it going to help a student and a 
parent to change if they know where they are? I think it will. The 
problem with too many classrooms in America is, people believe 
that the performance they are giving is going to give them the 
good life, and it isn't true, and I think one of the most important 
m^sages we need to bring to the students of America and to the 
parents of America is that what you are doing now is going to lead 
to failure in competition in that global marketplace, and if you 
really know that, we think you will change your behavior. 

I have strong confidence in the American public changing behav- 
ior if they are given adequate information, but I don't think we are 
giving them adequate information in education, and the issue is 
not tests so much as it is levels of achievement. I don't think we 
understand the levels of achievement we need to reach for, or, if 
we did, we would certainly have a different commitment to educa- 
tion than I see now in this country. 

Mr. GooDUNG. I would just take issue — and then I am finished — 
with one of your statements. I don't think we have had it good 
since World War II. I think the decline of this Nation b^an at the 
end of World War II with the breakup of the American family, and 
I am not bo sure that all the reforms we talk about are going to 
have a great effect unless we can find some other way to put 
Humpty Dumpty back tc^ether again. 

Chairman Kildee. Major Owens. 
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Mr Owens. Mr. Chairman, I want to point out that this has 
been one of the most informative hearings of this kind I have ever 
participated in. The participants also speak with great authority, 
and I hope that the record of this hearing can be made available 
very rapidly. We find it very useful in my sulKX)mmittee. 

I would nope that we could understand that there are at least 
three agendas, and probably many more, for education. One is the 
academic and pedag(^cal instructional agenda, one is the ^ial 
agenda, and another is the political agenda, which probably sits 
atop all that, and, whereas all three are very important, we can't 
have meaningful discussion unl^ we limit ourselves to at least 
one at a time, and I think what we should be addre^ing here is the 
academic, pedagogical, instructional agenda. If we can get that 
straightened out, maybe it will be helpful in some of those other 
areas; I am sure it will. 

Even youngsters in inner city schools, like mrat of the schools in 
my district in New York, would greatly benefit from more account- 
ability, and more structure, ana teachers who know more about 
what they are doing because they have a set of standards and there 
is accountability in the form of assessments that are more mean- 
ingful. I don't know what those are, although you say. Governor 
Romer, if you have the right test, it can be an invaluable instruc- 
tionaJ tool. I hope that as you develop the right test and that set of 
standards, not only do you have a cluster around that set of stand- 
ards that people can tie into and have different ways in which they 
deal with that set of standards, but I hope you have a cluster of 
standards, but testing for and educating for what? 

We are not always educating for academic achievement, to 
prtxluce scholars, and I think that is what our public school sys- 
terns are geared toward. Whereas math and science is emphasized, 
and that is a great universal standard, global standard, and you 
can measure that easily from one country to another, it may not be 
as important as we have made it out to be. Leadership skills— I 
hope that somewhere there will be a standard of education for 
leadership. You know, some kids who have leadership abilities 
don't do math and science that well nece^arily, although math 
and science can teach them to concentrate and the kind of disci- 
pline that will always be useful, whether they use it later on or 
not. 

But I would hope that those who have certain kinds of abilities 
are not thrown aside, but those who can perform well, sing well— 
you know, we have just throw^n out all of the art instruction and 
the music instruction out of New York City schools because they 
say it is not necessary, it is not vital. So without the motivatiun 
there, a whole set of kids are in more trouble than ever before. 

We are forgetting that one of the most effective political leaders 
of this century in the United States was an actor. I didn't agree 
with 90 f^rcent of his program, but I recc^ize his effectiveness as 
a political leader; all of us do as politicians. Ronald Reagan was an 
actor. Harry Truman was a small town lawyer. He was a lawyer, 
and there are a lot of lawyers around in Congress, but most of 
them are not lawyers who edited law journals and the Harvard 
Law Review. The leadership does not seem to come out exactly the 
way colleges and academic programs are structured. There are 
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great politicians who happen to be scholars, but most politicians 
and political leaders who are very good leaders are not scholars. 

So let's have a set of standards where we have a flexible set of 
standards, and we educate them, and know that the kid who has 
leadership ability may not do well in math. In fact, I understand 
most high school principals in New York City are ex-gym teachers. 
That may be bad; it may be good. 

A lot of the physical education majors that we used to laugh 
about and joke about when I was in college are making very high 
salaries in business now as salesmen and busing people. ^ they 
(ire obviously able to succeed in very difHcult areas. 

So I would hope that as we set this academic agenda— instruc- 
tional, pedagc^cal agenda — we will understand the kind of flexibil- 
ity where our system is probably ahead of the others in the world 
as being more flexible, but that ought to be, as we standardize, not 
lost, that we educate people for a number of different careers. 

The other thing is, I would say to Dr. Sewell, on the social 
agenda, the Committee on Economic Development recently issued a 
report where they said that the social agenda in the schools is as 
important as the academic agenda, because youngsters are seeking 
learning; they are not motivated because they are held back by the 
fact that they have so many different problems. We must address 
that social agenda, but I don't think we should put down the aca- 
demic agenda and the accountability, movement toward account- 
ability. That is necessary, and we can use it even more to justify 
attention to that social agenda. 

If kids test badly because of all th^ adverse conditions that 
they live under, let us deal with more of those adverse conditions, 
and I know that leads us to the political agenda, and you pointed 
out the fact that we are just not getting the kinds of resources and 
funds in the areas where poor kxos live, which may lead to Senator 
Brock's suggestion that we sue for the lack of constitutionality* 
that our present educational system is unconstitutional because it 
is so unequal and so unfair. 

So I would hoi» that a high level of professionalism could be 
brought to the whole discussion and maintained- One problem with 
education is, it always bogs down in these very emotional reactions 
and responses and we just try to cover all the territory at once, but 
you have done a great service by focusing on one aspect of it, and I 
hope we can straighten out the academic, instructional, pedagogi- 
cal agenda, and I wish you luck. Governor Romer, in your labor ne- 
gotiations in Denver. 

Mr. RoMEK. Yes. 

Co\\\d I reply? 

Chofirman Ivnj)£E. Yes, Governor. 

Ml- Romer. It is a very good comment, but, you see, if we are 
looking at fourth grade math, we ought to be thinking both of 
those who are going to be engineers and those who are going to be 
technicians out on the assembly line, because a technician needs to 
know how to problem-^olve, a technician needs to oi^anize informa- 
tion and say. This is solution one compared to solution two/* Math 
ought to teach one to do that, 

Mr. Owens. Problem solving is really what we are trying to get 
at. Math is one way to do it. 
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Mr. RoMER. Fourth grade math, the test, the assessment instru- 
ment, and the curricula that relate to it ought to he teaching us to 
compute, but also ought to be teaching us what we would call 
higher-order thinking skills, which are the problem-solve, and I tell 
you, the factory worker needs that, also, in addition to the i^rson 
who is going to be an engineer. 

When you get to the llth grade or the 10th grade in high school, 
one mav say, *'rm going to take calculus and take a particular vo- 
cational course/' Another may say, "No, I'm ^oing to do something 
else and not take that course/' But the critical question that I 
want to pose is this: I am the parent of seven children. I need to 
know when the youn^ter is in the fourth grade, is that child ill 
educationally or well educationally? 

If that school and that toacher can't give me an accurate report 
on that, they have failed me, and if I, as a parent, don't take that 
information and. Congressman Goodling, I agree with you, the 
family has to take more responsibility, if we don't take that infor- 
mation, we have failed as a family. But I toll you, as a society, we 
fail unless we get a system in place that rings that bell at the 
fourth grade and says, "We are going to give you an accurate 
standard to which you reach. We are going to give you a way to 
test where you are and we are going to give you some remedial 
action to change the education of that youngster.*' 

That is what we are after. 

Chairman Kildee. Mr. Petri, 

Mr. Petri. Thank you. 

I learned from the morning paper why my colleague. Major 
Owens, has greater respect for actors than perhai» you once did 
since you have a leading actor in your familv who is doing very 
well He may end up being President some day with that acting 
background. That has probably softened you up on at least this one 
old actor 8 retired- 
Mr. Owens. He hates politics, he will never make it. 
Mr. Petri. Okay. 
(Laughter] 

Mr. Petri. Sitting here, I guess I have a couple of questions. One 
is of either the Governor or Secretaiy, Is the assumption that we 
don't have adequate or good testing for tracking purposes, or is it 
that people aren't acting effectively on the information that is 
being pnxluced by the testing that is now being done? 

I understand it is being done in 38 States. Are there any where it 
is being done adequately or are they all failures? Is testing the 
magic key or better testing, or is it acting on information being 
produced by tests we have already? 

Mr. RoMER. Could I take a crack first, and then Mr. Brock? 

It is both. We do not have adequate t^ts. Too many of our tests 
are norm-based tests, the standardized norm-based test that com- 
pare ourselves to each other and we are not adequate. We are not 
cluing ourselves to a worldwide standard. So part of it is the test is 
normative and it is on a bell curve, rather than criterion-based* 
which is one I illustrated for — this is what you need to know and 
be able to do. 

Secondly, the tests themselves are sometimes inadequate. Multi- 
ple choice tests are always based upon the assumption that there is 



195 

a right answer and somebody else has already posed the question. 
If you train a generation of scientists on that, vou know, you are 
going to have a bad group of scientists tecause the challenge in sci- 
ence is to adk the right question and to understand that there may 
not be one correct answer. But it is not just the test, it is a problem 
of we are not acting upon the information that is there now and we 
need to have something that brin^ it more vividly to our minds. 

Mr. Brock. Just a little elaboration. We lest pretty well for 
people who want to pursue an academic career. We don't test at all 
for anybody else. We really don't know how our kids are cooing in 
relation to what they have to be able to do in the early grades, and 
that is what I think Governor Romer was talking about. 

He talks about the fourth grade, but at any stage in that first six 
or seven years of school, those are the years that you are going to 
make it or break it and if we don't know at that stage ho ^ we are 
doing against a standard that, in my judgment, has to be globally 
derived, we don't know how to change what we are doing or how to 
adapt or how to respond. 

We underestimate our teachers. They are hungry for this kind of 
information; they are desperate for it. They are eminently capable 
of doing what is necessary if they are given the tools, but what we 
do is we say, these are the tools you have to do if you want to get 
somebody to go to the University of Tennessee. 

Mr. Petri. The nice thing about education is that everyone who 
is an adult is an expert because they have all experienced it from 
the point of view of being a student. All I can remember is that I 
was tested in public schools in Fond du Lac, Wisconsin, from morn- 
ing kindergarten on, and I thought they were fun. I thought they 
were games when I was very young. There were hearing tests and 
eve tests and recognition tests and 101 different kirds of tests all 
the way through. Are we talking about what is already being done? 
You, sin are in the business and mayte you could share with us 
your thoughts on whether the problem is really figuring out tetter 
ways of assessing or acting on the information more efTectively? 

Dr. Anrig. Th^k you» Congressman. It won't surprise you that I 
think we know how to make good tests. The issue is not whether 
we know how to make a good test, but do we know what we want 
to make the test for, and that is why I telieve, and each of the 
speakers today has said, first you have to decide what are you 
trying to accoiiiplish? What is it you want a youngster to know and 
be able to do? Then you design a test to measure that. 

As Congressman Goodling said, anyone can~I forget the expr^- 
sion used, but I don*t think I would want to put it back on the 
record, but we know how to measure knowledge, but we can't and 
shouldn't be the ones who define what that knowledge is. That 
ought to be designed through a very broad-based consensus process. 

Mr Brock. Tom, can I just add one point. In today s workplace, 
there is a radical difference in how we organize work and the 
really good companies, Motorola, Xerox, firms that are way ahead 
of the curve, that are doing what other firms are doing around the 
world, and they are becoming very competitive. If they are not re- 
organizing work, by the way, they are not stajdng competitive and 
we are trying to compete on the basis of wages, but tnose compa- 
nies that compete on the basis of skilb are finding out that they 
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have to focus on qviality and they have to focus on teamwork and 
they have to focus on horizontal organization, which means work- 
ers individually have to carry more rraix>nsibi]ity. 

Now, we are not teaching those kinds of skillst nor are we meas- 
uring those kinds of skills and you can start teaching team process 
as pirt of a math session in the fourth grade. But it requires some- 
body to tell the teacher, this is what we have to have to be globally 
competitive and we are not telling them that now. That is the 
point. 

Chairman Kildee. Mrs. Unsoeld. 

Mrs. Unsoeld. Thank you, Mr. Chairman. 

I came in here sort of open to learn and I found myself fast be- 
coming a fanatic. 
[Laughter.] 

Mrs. Unsoeld. Governor, I share your fervor when you spoke 
that when the bell rings at the fourth grade, we need to do some- 
thing, but I want to submit, **the bell rang a heck of a lot earlier.'* 
We have used some Persian Gulf examples and the one I would use 
is that when the Patriot was begun to be develop^, we had in the 
United States 70 percent of the world manufacturing of consumer 
technology and we are now down to 5 percent, so I think we all 
agree, Mr. Brock, our nation's economic future in international 
competitiveness defMjnds on improving our country's educational 
system. We have a problem. 

But what I hear is that more accountability is going to cause cor- 
rection to be made. Again, Governor Romer, you said that knowl- 
edge of where they are on that graph ought to motivate them and. 
Secretary Brock, you said let the student know where he is and 
then some correction can be made. 

But both of those statements assume that there is motivation 
and they assume that education is beginning at about the fifth 
year of life, and it isn't. It is starting way earlier and we know that 
the kids coming into kindergarten are much less prepared to suc- 
ceed today than they were lO years ago and 20 years ago, so the 
bell has already rung and we have been warned and we are not 
making that correction yet. 

Probably by age two or three, the motivation for learning is 
pretty well established and any great quantum leap forward you 
may make after that is overcoming an extraordinary handicap. Wc 
have to aim it at that earlier age. 

We know that the investment there is going to pay off. We know 
that when we find pregnant teenage girls and we get them into a 
special kind of high school class, that the oiT>^pring produced there 
are going to perform better when they become school-age. We know 
we can make that investment, so the bell has rung, and gentlemen, 
it is not assessing it after we get to school. I^t*s learn what we al- 
ready have and let's concentrate that fervor that you have to 
m..!king sure that we are paying for those early childhood educa- 
tion investments, we in this country. If we fail to do thin, our 
nation will not make it in this next century. 

Mr. Bkch k. Mrs. Unsoeld, if I may just jump ahead of the Gover- 
nor for a second, neither one of us believes that testing or assess- 
ment is anything more than a very partial part of a solution. For 
myself, and I think I would probably speak for almost everybody 
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here, if we don't start dealing with the front-end problem, we are 
never going to solve it with the back-end solution. We have to deal 
with pre-natal nutrition, post-natal nutrition. We have to deal with 
early childhood education. All of those issu«3 arc fundamental to 
rorrecting the problem, but no sii^le answer is going to deal with 
this problem and what we are saying is that tockiy we are talking 
about one aspect of it that really has to be addre^ed. We do have 
to know where we are. 

The teacher has to know where they are. They have to know 
what thev are being asked to do and we are not telling them. 

Mrs. Unsoeld. I submit that the teacher knows that without a 
national assessment, but the teacher is so having to concentrate on 
making up for the failure, as Mr. Goodling put it, of our families 
having become dysfunctional, that they are having to try to 
remedy for what was lost in those early years. 

Mr. KoMER. Could I make a comment? 

Mrs. Unsoeld. Yes. 

Mr. RoMER. You are abwlutely correct. The panel which I chair 
has the obligation in the next two months to figure out what are 
the five or six most critical auctions to ask each governor to 
report on in terms of goal number 1, every child shall be ready to 
enter school by the year 2000. The posing of those questions, Mrs, 
Unsoeld, are really critical and I understand that. 

But let me come back to the Federal establishment here. What 
we have talked about this morning is a failure of research and de- 
velopment in assessment in this nation. We have been talking 
about it for two hours. We have not done the kind of research and 
development in terms of the standards or the assessment instru- 
ments or how we use them. 

Now, let me ask you, since we have been talking about the Patri- 
ot missile, go back and compare the amount of dollars that have 
been spent in the last 20 years by the Federal Government on re- 
search and development on weaponry as compared to research and 
development upon what are we doing with assessment and goal set- 
ting in our educational system? 

If you want a classic failure of allocation of resources to the right 
area, look at your own budget in terms of research and develop- 
ment as it relates to education. 

Mrs. Unsoeld. I join you in your fervor and let us combine it and 
try to make that repriortization in our funding. 

Chairman Kildee. Mr. Hayes. 

Mr. Haves. Thank you, Mr. Chairman, 

I am really caught up in this testimony very much and I am very 
much a stickler to tinje. I realize the difference between the green 
light and the red light is only five minutes and I have to ot^rve 
that. 

But I just wish we had an opportunity to have some of this testi- 
monv heard by the entire 435 member body which I am a part of. 
Maybe we would get serious about reforming our educational 
system. 

When you talk afcK>ut making it our prioritVt it sure isn't. We just 
spent $?0 billion yesterday as a part of the bail-out program and I 
understand that on track is another $50 billion^ 1 guess, by Septem- 
ber. Yet, when you look at the amount of resources we are willing 
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to invest in our educational system, it seems to be such a struggle, 
you know. 

I happen to represent, as my good friend Goodling has already 
indicated, part of the City of Chicago, which happens to be the 
poorest, a sch(K>l district that the amount of money spent per stu- 
dent is much lew than the amount of money spent on the student 
that surrounds Chic^o in the suburban areas. 

The main reason for that is the tax base that is used in order to 
fund the schwl system. Illinois, as a State, doesn't have a good 
record as to how it disperse money to the various school districts. 
What they use is a property tax base as a means of doing it. There 
is no property— if the value of property is down, the amount of 
money you get for the schools is down. 

We got, I guess, the highest dropout ratio of any part of the State 
of Illinois, It runs about 50 j^rcent in many of the high schools in 
my district between the kid who starts the high school as a fresh- 
man and one who stays long enough to get a diploma. 

Testing is not going to solve that problem. I agree with much of 
what you said about we ought to have assessments at different 
levels as to how a student is doing and the check with the faculty, 
the teachers and the parents. Parental involvement is very impor- 
tant. 

I do want to ask a question of you, ^nator Brock. I have reached 
the point where I have almost thought that we are willing to write 
ofT a lot of kids because of the economic status. You don*t care 
whether they get an education or not, foi^et them, you know, even 
as we talk about choice, and this is a big issue. What is going to 
happen to that poor kid whose parents can't choose what school 
they go to. That is the school that i^^ going to get the money. Part 
of the criterion for getting in that particular school is going to be 
your grade level or your achievement. 

What is the impact—this is my question— for students who do 
not achieve the standards that you would establish by age 16? Will 
we provide other opportunities for them to achieve or would we 
begin to track students at an early age in the areas that they per- 
form best? How do you deal with that kind of problem? 

Mr, Brock. What we suggested. Congressman, in the Commission 
on Skills of the American Work Force, was that we, as a society, 
accept that as our responsibility, and Dr. Anrig was right, we are 
not talking about putting all this burden on the students. It is our 
burden to get them up to a level of productivity that is globally 
competitive. 

W« said that we ought to try to commit to get everybody up 
there by 16, but as Governor Romer has said, not everybody rnoves 
at the same pace. Some move slower, some move faster. 

If they make it by age 16, then we have opened up a whole set of 
choices for them that they can move to. If they haven't— and let's 
admit vhat there are some kids that can't make it in public school, 
for whatever reason, family. Congressman Goodling is absolutely 
right, there is a fundamental family problem in this, but whatever 
the causal factor, there are some kids that aren't going to make it 
in the public school setting. 

So what we said is, look, that doesn*t relieve us of our obligation. 
We need them as participants in this society. We can't constantly 
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carry 30 percent of the public of this country on our backs. Every- 
body has to have an opportunity to be productive, so let's set up 
alternative learning centers and let me tell you about one, because 
we already know that this thing works. 

In San Di^o, there is a guy named Tony Trujillo, that runs a 
public »:hool. It is the biggest district in the United States. In that 
district, they have decided that the money follows the child and if 
the child drops out of public school, they have an alternative pro- 
gram for them that they put them back into some form of educa- 
tion. The fact is it is in the same school community because they 
think kids like to stay with their peers, but it is a different process 
entirely. It is radically different. 

Do you know what the average daily attendance of dropouts is, 
these 2000 dropouts, is 97 percent. That is higher than it is at the 
Naval Academy. Eto you know that they are out-performing the 
students in the regular classes? Do you know they are learning 
faster and more than they do in the regular classes? We know 
what works. We are not doing it. That is the point. We know we 
can do it. We know we have to do it. It is insane to think we can 
tolerate 30 percent of the kids in this country falling olT by the side 
of the road. This country can't compete with anybody if we don't 
educate those kids and we have to accept the responsibility of 
doing it. 

We can't do that, though, unless we know they have a chance of 
falling off the track and that is where the awe^ment comes in. It 
is just a limited part of the answer, but it is part of it. 

Mr. Hayes. All right. 

Chairman Kildee. Dr. SewelK you wanted to add something? 
Mr. Hayes. He is itching to do so. 

Dr. Sewell. 1 think the notion of an alternative learning center 
is a very desirable one if we can identify some of the strategies to 
nmke it a valuable one for the kids, but that is one of the areas 
that we have to be very cautious about when we think in terms of 
assessment because the history of testing in America has been used 
for tracking, sorting and putting individuals who are capable of the 
highest intellectual performance into those tracks. We think in 
terms of a State that is testing kids in kindergarten to determine 
who is ready for first grade. 

If we start that tracking system and use standardized tests to ac- 
complish that goal, we will start the weeding process very early 
and undesirably so. 

Mr. Hayes. Some of us call it creaming. 

Thank you. My time is already gone. 

Chairman Kildee. Mr. Koemer. 

Mr. RoEMER. Thank you, Mr. Chairman. 

I would just like to thank the distinguished panel. I planned to 
leave about an hour ago to attend another committee hearing, but 
this has been so fascinating that I chose to stay. 

I also hope. Governor, that when you iraue the report card from 
the National Governors Association you give the same attention to 
the Congress that you do to the White House because we must 
work closely together on this problem. Congress is very, very con- 
cerned about this issue. Maybe you could come up and present the 



EMC 



200 

report card to us and work with the chairman on an idea of that 
sort. 

Coming from Indiana, I would just like to sav that the education 
problem is the most critical one that we face. We all like to make 
the analogies to the Persian Gulf War or World War 11. But educa- 
tion is the essence of our society and our children's future. In a 
rural county in my district, the dropout rate is 25 percent. The Di- 
rector of Correctional Facilities said to me the other day in Indian- 
apolis, "The biggest indicator of prison space in the next 20 years is 
the nun^ber of at-risk kids in second grade/* 

That is frightening thought for us as a community and as a coun- 
try. Yesterday I heard t^timony from another distinguished panel 
that our facilities in colleges are failing apart. In 1991 we are 
spending in constant dollars for instrumentation and facilities mod- 
ernization half as much as we did in 1966. We have tremendous 
problems which need America s attention. We must work together 
to solve this problem. 

My question is, given the limited resources we have, and the lim- 
it«i capital, where would you place national testing in terms of the 
scope of the problem that we face? 

Mr. RoMER. Could I give my own personal answer? I have, in the 
back of my mind, about 17 things I think we need to do to reform 
education. This is one of the 17, so it is not the full answer. 

Mr. RoEMER. Where does it fall. Governor? 

Mr. RoMER. The matter of setting standards falls right at the 
top. If we do not determine what it is we are trying to do first, all 
the other 1(5 reform elements in mv mind don't iit. I would say not 
testing, per se, but arriving at the levels of achievement, a national 
consensus of what it is a youngster needs to know and able to 
do, I believe ought to be the number one item in terms of the 
timing in which we approach it. 

Now, it isn't— teaching und what happens in the classroom is fi- 
nally what is really the most effective thing that makes a change 
in education, but I don't think that we are clear yet about what are 
the levels of achievement that we need to reach for and I would 
ust say, it is one of the very top iten.s, but it is only one of a long 
ist of educational reforms tnat we need to accomplish. 

Mr. Bkcx:k, The problem is, if you do any one of these things, you 
may do something that is counterproductive because they are 
hooked together. I mean, people talk about choice. If you have 
choice without school base management, you are probably making 
a mistake. If you try to have school base management without 
giving the teachers the assessment tools, you are probably making 
a mistake. 

These things link together in a coherent whole. The reform of 
public education has to go across the whole range of subject mat- 
ters and it isn't a matter of resources. Congressman. We— Lord 
knows, if you think education is expensive, try ignorance and just 
see what this country is going to do if we try to compete on the 
basis of lower wages all the time. 

We don't have that luxury and it isn't a matter of spending 
money. All the research that is being done in assessment is being 
done in the private sector. Where in the world is the Federal Gov- 
ernment? WV don't do any R&D and that is one role that this gov- 
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ernment ought to be able to do at a very low cost and high ratio of 
yield. 

Mr. Owens was touching the right point earlier. He was right on 
the money when he said, we have to teach reasoning skills, logical 
skills and that means different things to different kids. But we are 
not saying a test is goii^ to answer. We are saying that if you (x>m- 
pile a seri^ of evaluations over a period of time, you are giving the 
benchmarks to that student and to that tocher and to that parent 
and to that principal that they have to have to move in the right 
direction affirmatively and if you don't have thc^ benchmarks, if 
you don't have those milestones, we don't know where we are 
going. 

Today, we just don't have them. 

Dr. Anrig. Let me, if I may, just add one further point. I want to 
strongly endorse the points that have just been made by the Gover- 
nor and Secretary Brock. You can't look at assessment as some- 
thing separate unto itself If you do and you act just on that one 
piece, you are apt to foul up the whole deal. 

What we have to do, and I agree, the absolute first step ^^ust be 
clear up the standards, then have some way of measuring how we 
are doing on that and then, Congressman Petri, be sure the kid is 
getting the help and the instruction that makes it po^ible for him 
or her to succeed with those standards. 

If you do them through the latter part, don't even bother doing 
the first two. 

Mr. ROEMER. Mr. Chairman, may I follow up for just a moment? 
Chairman Kilde£. Certainly. 

Mr. RoEMER. &cretary Brock, since you have a great deal of ex- 
pertise with the Americas Choice study and the Work Force 2000 
study, what specific recommendations would you make in vocation- 
al education? How should we involve parents and the business com- 
munity in this aspect of education as well? 

Mr. Brock. Let me say that I almost feel like we ought never to 
use the word vocational education again. All education ought to be 
training you to be productive as a human being. That is sort of vo- 
cational in the purest sense of the word. 

What we are trying to do with the Secretary's Commission on 
Necessary Skills, the ^ANS Commission that Elizabeth Dole 
asked me to chair, is to see if we can work with labor and with 
business to describe what we call functional skills that are neces- 
sary to be employable and functional as a human teing, and with a 
high growth path. 

So we are saying if you are going to teach math— and math is an 
enabling skill; you do have to be able to make change in restau- 
rants; you have to be able to count a tip or pay a taxi; there are 
certain basic things. But shouldn't we be teaching math in that 
context as a functioning skill, not just as a numeric column of 
numbers? 

We do the worst job of an industrial nation in the Free World of 
communicating to students the relationship between what they 
work on in school and what they are going to work on in life. The 
school-to-work relationship is not established in this country, and 
that is part of the problem. 
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So your question on vocational may be precisely the right ques- 
tion. Can't we reorient our education so that we are creating a 
sense on the part of the student as to why they are learning and 
how it would work for them, what it would mean to them? Can't 
we teach them functional skills as a part of that math, as a part of 
that science, as a part of that English training? If you do that, then 
you are teginning to create the linkage that really dora give a 
sense of motivation and l<^c as to why they are there. 

By the time the kid reaches 16, if they haven't dropped out al- 
ready, they are Iwred silly for the next two years, 90 percent of 
them Why? Because they don't know why they are there, and, 
frankly, business doesn't either; there are not five businesses out of 
a thousand that ask for a high school transcript. What is the mes- 
sage? "It doesn't mean anything. All we want to know is, will you 
be at work on time?" 

Mr. RoEMER. My question is, how do we get those businesses in- 
volved in the schools and in that kind of training? Business repre- 
sentatives come to me saying they want to be involved. 

Mr Brock. If you can get businesses to help us describe what 
those functional skills are, then you measure those skills and you 
come up with a series of what we call merit badges or certificates 
that a business can take and evaluate, and all of a sudden you 
have created that linkage where the business has a vested interest 
in the product of the school system, and you are creating a sense 
where the business has an investment in trying to make it better. 
If you don't create that linkage, you are not giving the system the 
pull that it has. We have got the push coming from underneath, 
from the kids. We need the business community to pull in the right 
direction. That is sort of the priority we are tr\ ing to establish in 
the SCANS work. Can we motivate the business community to tell 
teachers for the first time what they want? If they do, we can de- 
scribe that, then we will have to learn to teach iU then we will 
have to learn to measure it, and then, all of a sudden, we have got 
a system that a young person can understand and a b tsiness can 
understand, and it begins to compute for *hem. 

Chairman Kildee. Governor Romer. 

Mr. Romer. I know we are getting near the end of our time, and 
we have just spent two hours describing the problem. There is a 
way that we can go at this jointly, and I would like to take a 
minute or two to just describe that. 

If we get our minds together — Congress, the executive branch, 
the governors of the Nation, and all the other educational interests 
that have something at stake — we can go at this, and we say, **Hey, 
what we need to do first of all is to arrive at levels of achievement 
in the m^jor subject matter areas.** That can be done. It can be 
done on a collaborative, a consensual basis. 

On math, you start with those who know it best, like the nation- 
al counselors, the teachers of math. You take what has been done 
to date at the national level, and you distribute it to the Nation 
and say, **How do you feel about that?" You get their reaction. 
Then you come back, and you arrive at a standard, and it can be 
one which can be changed from time to time, and then, systemati- 
cally, you l^^n to do that with other subject matter areas. 
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We arrive in f Nation at what specific levels of achievement 
are needed, and men that is a portion that is done. Then we turn 
to the question of ass^ment, and we say, **Look, you in the States 
have a lot of resource already you are spending there. Let's help 
you combine so you can get more bang for the buck," 

CongreM could come to the table and say, **Look, we will put 
some R&D money on the table to help do this, but rather than us 
do it at th^ Federal level, that's wrong, we will do it with you." 
You could put some matching grants on the table, and you cou^d 
design a program in which you could aim at assessment that 
dwsn^t just hold a report card up, but assessment, as many o^ you 
have saiH, changes the system. 

Then we can go from there, from standards, to assessment, to 
looking at, all right, what are the other 16 educational reforms? 
and then let's get a partnership approach to that from the Federal 
and State level. 

But as a motion — and this is what the educational ^oals are all 
about "'ve ought to say, if we really believe in that, let's put it 
right at the top of our priorities, let's get a program that is 10 
years in length, and let's start down that path. But we ought not 
just sit here this morning and say, "'Hey, we can't get there from 
here/* We can, but I think the way to start is what we have been 
covering this morning, and that is to start with some consensual 
understanding of what it is we are trying to achieve, a level of un- 
derstanding of what a youngster should know and be able to do in 
various courses at various age levels. 

Mr RoRMER. Governor, I think that is a great idea. In my home 
State, they are starting to lay off teachers. We all get up in front of 
our microphones and say hov important education is, but we must 
come up with some ideas fo» these partnerships and teamwork be- 
t'iioen our States. I hope we can do that before the report card. 

Chairman Kildee. Mr. Petri. 

Mr. Petri. I just have one quick follow-up question. 

You focused on dropouts and people who don't meet the stand- 
ards. What about the people who very easily meet them, who 
aren't challenged enough? Is this a national problem, that we are 
underj^rforming because we are not utilizing resources out there 
of people who, with no problem, do the test in half the time, and 
they are ignored somewhat by teachers because there are other 
kids who have problems? They get bored. Are we losing as a coun- 
try in competing because of not demanding enough? And, if we 
have national standards, isn't there some danger that they may be 
set too low because we will not want to embarrass i^ple, that we 
are trying to figure out a kind of reasonable thing where most 
people will be sufficientlv challenged* and therefore we will set it 
there ^ that most people can be engaged, and not set it at the 
l^vel that will challenge people in the top 20 percent or so? 

Mr. Anrig. Congressman, you raise a very key point. We should 
not set standards in such a way that there is a single fixed point 
that we expect all kids to reach, because that is exactly what you 
are going to get if that is what you set. What we ought to have an.* 
striving standards, maybe levels of thenu if you will, so that a 

{roungster ^1 feel that he or she is making prc^r^ towards a 
evel Dut abo c^in go well beyond it. 
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The National Amassment Governing Board has got three levels 
in the standards that they are working on now — Iwisic, nroficient. 
and advanced — so that youngsters could have sonre feeling of move- 
ment and schools could have some feeling of movement, and not 
just a single, fixed point. 

I think we have got a mentality in the United States that we 
want to put on the front page of every newspai^r one number that 
covers everything, and that is part of our problem. 

Mr. Brock, One of the things that we have wrestled with in the 
SCANS Commission is precisely this question. The one thing we 
are desperately trying to avoid, and being very sure we are going 
to avoid, is a pass/fail kind of approach. That is not what we are 
about. We are talking five levels, but the same kind of a concept of, 
nobody fails, nobody exceeds, but there is a range within which we 
do expect, and I think we can use the system to motivate people at 
every level if we do it right. 

Chairman Kildee. Mr. Goodling, for a concluding remark. 

Mr. Goodling. Just two. 

Some of the finest research that has helped education and also 
the medical profession has come from the military research^ and so 
that wasn't all bad. 

If I sounded like I was getting very excited about this issue, 
abc it reform, I am, I get excited because my concern is this. In the 
middle of this panel we have two very successful, very outstanding 
gentlemen who should be driving the people who are flanking 
them. My fear is, as we go through this reform process, that two 
gentlemen times 100 gentlemen just like you— and women, very 
successful, very outstanding — will try to drive this road alone, and 
we will not have the gentlemen fianking you in that automobile, or 
the four that we had yesterday testify. That is my greatest fear 
about the whole reform movement. 

You both are in a position, because of who you are and the suc- 
cesses you have, to drive. Perhaps the two who fiank you aren't 
quite in that kind of position when trying to drive the public. But 
my fear is that those like you may try to drive this road alone, and 
from a lot of the reform movement that I have seen, that has taken 
place. That is doomed, in my estimation, to failure. 

If you can drive the two who are flanking you, and if they can 
drive with you, then I think we can come up with the kind of 
reform in this country that we positively must have. 

Mr, RoMER. Cungref^man, that is very good, and you and I have 
talked privately about this. 1 want you to know. 1 am responding to 
some of your thoughts 

The goals panel in the month of April is going to go to eight 
places in this Nation and lay out where our thinking is to date and 
the expertise that we have heard to date and ask the gentlemen in 
education who are on site, doing it, to share with us their criticism 
and their input, and I think that all the way along in this goals 
panel project we really do need to have it very inclusive, very in- 
clusive of the people who have a stake in it. and particularly those 
who are on the front line, the teachers. We really need to make it 
inclusive, and so I take your comments seriously and have incorpo- 
rat4Hi it in the life of the panel. 
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Mr, Brock. I think it is a super point, and I couldn't agree more, 
but 1 think you overratimate the two of us. I would just like to be 
in the education car; I don't have to drive it; I just want to contrib- 
ute somehow to moving this thing, because I am truly, truly wor- 
ried about this country right now. I don*t believe we can compete 
with an uneducated work force, I really don't, and I don't think we 
are doing an adequate job by our children. 

Mr. Hayes, Could I raise just one issue? The Senator caused me 
to have to raise this final question with your comments now. 

When do you plan to submit your final SCANS Commission 
report? 

Mr. Brock. We are going to have our first phase of the report, 
hopefully, in May, and then we will try to refine that over the fol- 
lowing six to eight months after that, and have a completed prod- 
uct by early next year. 

Congressman, it is our intention to set down the basic bench- 
mark of this effort within the next two months. 

Mr. Hayes. That is some pretty fast tracking. 

Mr. Brock. Well, it is faster than I thought possible, but the sur- 
prising thing to me has been just how active people have been in 
participating in this effort. We have had really astonishing support 
from labor, from business, from educators, and. frankly, we are 
moving faster than I thought we could. 

But understand that what we do in May is going to be very ten- 
tative. We are going to put it out there and say, "Kick our teeth 
in/* and we will get a lot of people who will try to do that. We will 
then reshape it and reform it and try to adapt to those criticisms 
and see if we can make this thing something that is workable. I 
hope it will be. but it is a big task. 

Chairman Kilx>ee. I want to thank the panel. I am not exaggerat- 
ing in the least when I say that we really have xissembled one of 
the finest panels that I have had the privilege of listening to in my 
political career. We have had concurring and contrasting views. I 
think Bill Goodling. as he always does, gives good advice; I think it 
was very good advice and you all accepted it very, very well. 

What I intend to do— because I think this hearing has been very, 
very helpful— 1 intend to put the printing of this record on a very 
fast track so that the educators, and elected officials, and the 
public will have the benefit of your very insightful testimony, and 1 
fH»rsonally thank all of you for being very, very helpful to the 
Nation. Thank you very much. 

Mr. Brock. Thank you very much. 

Chairman Kildee. Our next panel will consist of Dr. Saul Coo- 
perman. president. Educate America, Morristown. New Jersey: Dr, 
Karl V. Hertz, superintendent, Mequon-Thiensville School District. 
Mequon. Wisconsin: Dr Monty Neill. associate director. FairTest. 
Cambridge, Massachusetts: Dr. E.W. Kelley. professor of American 
Government. Cornell University, Ithaca, New York; and. an old 
friend, Gordon M. Ambach. executive director of the Council of 
Chief State School Oflicers. Washington. DC. 

Dr. Cooperman, you may lead off. unless you have made sonip 
other arrangement among yourselves. 
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STATEMENTS OF SAUL COOPERMAN, PRESIDENT, EDUCATE 
AMERICA. MORRISTOHTV. NEW JERSEY: KARL V. HERTZ. SU- 
PERINTENDENT, MEQUON-THIENSVILLE SCHOOL DISTRICT, 
MEQUON, WISCONSIN: MONTY NEILL, ASSOCIATE DIRKCTOR 
FAIRTEST. CAMBRIDGE, MASSACHUSETTS: E.W, KELLEY. PRO- 
FESSOR OF AMERICAN GOVERNMENT, CORNELL UNIVERSITY, 
ITHACA, NEW YORK: AND GORDON M. AMBACH, EXECUTIVE DI- 
RECTOR, COUNCIL OF CHIEF STATE SCHOOL OFFICERS. W ASH- 
INGTON. DC 

Dr. CooPERMAN. Mr. Chairman and members ot the committee, I 
am Saul Cooperman, president of Ekiucate America. It is a nonprof- 
it, nonpartisan organization in Morristown> New Jersey. ^ am here 
to represent former New Jersey Governor Tom Kean. 

I heard two comments of the former panel and of the representa- 
tives that I want to comment on first, tecause I hope the baby isn't 
thrown out with the bath water. 

First, local control— that if we have any sort of national assess- 
ment system or national exam, we are hurting our local control. I 
have great respect for it. I have been a teacher, principal, and su- 
perintendent for 23 years before I was Commii^ioner of New Jersey 
and my pr^nt position now, so I have respect for local ingenuity. 

But I would ask the panel and I would ask everyone to just con- 
sider, when v/e keep saying local control, do we realize that we 
have a national textbook system and four or five textbook manu- 
facturers say what is taught? The teachers are trained essentially 
the same way by departments of education and the deans. We have 
reciprocity in almost every other State. The school vear is almost 
the same, and the vacations are the same. If I blincxfolded anyone 
and put them in Arizona, or Alaska, Missouri, or Michigan, in the 
fourth grade, they would see essentially the same schools. So what 
I am asking you to consider is, when you consider national assess- 
ment, don't mix and match the local control issue. 

The second, the deep problems in our cities. Congre^woman Un- 
soeld and Dr. Sewell, I thought, spoke rather eloquently. There are 
no easv solutions. I spent four days each week in the city of 
Newark, New Jersey, trying to fmd solutions. There are not easy 
answera. But because we have problems in other areas does not di- 
minish the need for national assessment. 

We believe we should have required national achievement exams 
for all high school seniors and all fourth graders and all eighth 
graders, not some, not a sample, all. Such an examination — and I 
will focus on the seniors— would provide a reliable accountability 
system to students, to their parents, to schools, and to the States 
for the first time in our history. 

I believe we should have a core curriculum. We ought to state 
what children should know and determine whether they know it or 
not. Therefore, I am in agreement with almost everyone on the 

E receding panel. If kids don't know what they are suppc^ed to 
now, we have got to challenge schools. 

Someone asked the question^ what do we learn from the exam? 
Well, a very simple tmng we would learn from the exam. If the 
teachers aren*t trained to teach what they ought to» we can drive 
the results back to the teacher collies. We found that in New 
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Jersey. We gave a test and found out that in one of our colleges 50 
percent of the teachers could not pass a test in their academic 
m^jor; their grade point average was 3.2. The rigor was diminished, 
the grades were inflated, teachers were pa^ed out, and they 
couldn't teach their subjects. A national examination would key 
that, 

I want to give you a concrete example. I would like to give more, 
but I will give one of what a national achievement exam will do 
and create things for the better. 

In many of our high schools, when the kids don't attend college, 
they know a diploma is important; they know employers are going 
to ask, "Do you have a diploma?'' They are very savvy. The know 
the diploma is important, but that is it. In the last 25 years — and 
Ernie Boyer points this out in his book **High School/'— the gener- 
al track nas increased dramatically; more kids are taking general 
science and general math than biol<^ and algebra, and why not? 
They all count for five credits. So as long as you get the credits, 
you get the diploma, and the employer asi^ for the diploma. 

We have given the message to students that taking tough sub- 
jects does not count. We have got to deliver the mes^sage that work- 
ing hard and achieving means something, and there will be some- 
thing else. You talked al^ut linkage, and the other gentlemen 
talked about linkage too. Here is real linkage. If we link accom- 
plishment in school to the world of work, then you have got a hell 
of a connection. The message should be: Do well in school, and you 
will get a better job at better pay than if you graduated with a me- 
diocre record. 

Until now, there was no way to make these words and deeds the 
same, but because the President and the governors have estab- 
lished a consensus on goals in reading, writing, math, science, his- 
tory* and geography, we have the willpower to take the next Ic^cal 
step and ask with seniors, and then eighth graders and fourth 
graders — 1 would do it the reverse way — we will have a state-of-the- 
art system that testing people — Greg Anrig has told me, and others 
have told me, **We know how to do that, Saul. Set the goals, set the 
standards, get a national consensus, create the exam; then we are 
operating from a basis of fact, we are operating from a basis of real 
data, data that we don't have before. 

This does not mean top down. As everyone so eloquently stated, 
it can mean real inclusion. We can get the people who have the 
credentials, let everyone in, hold meetings across the United 
States, let anyone come forth, make it inclusive, great involvement, 
but have a national examination. Then business people will ask 
student^*, **So huw did you do? What do you know?'' And we at Edu- 
cate America have a five-point scale; I could talk to that later* if 
you wish. 

The National Achievement Examination has the power to 
change things, and it will galvanize the public, it really will. 

Governor Romer was talking about, you have got to have an ac- 
curate picture; this will give an accurate picture. And Governor 
Romer said the man on the street must understand it. One can be 
simple without being simplistic; one can direct and understood. 
We can give a national examination. 
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Right now, mOTt people in this country pay little or no attention 
to international comparisons or NAEP exams. I was on NAEP for 
two years as the rommi^ioner s repr^ntative, and then when it 
became the National Assessment Governing Board I was on for tw^ 
years* I used to go home in New Jeney, and I was all fired up and 
very upset, and I would get teachers, and administrators, and 
board members, and parents together, and I would say, "We're not 
doing very well; we really aren't; the data is there;" and they 
would say, ''Saul, what are you getting upset for? What are you 
getting upset for? Our schools are doing ol^y. No one knows; tney 
don't compare what we are doing with Osaka or Munich. Why 
should you worry?'' You s^, they didn't worry, because it was a 
sampling of students in the United States; it wasn't brought back 
to their door, it wasn't brought back to their school, it was kind of 
antiseptic and anonymous. Because the results were based on a 
sampling of students, the menage never came to the specific 
school. 

With a national achievement examination, results would be 
known for every high school and every high school child in the 
country. The message of results, the m^sage of facts, would be 
publicly delivered to every high school door, and facts are stubborn 
things. For the first time in our history, we would have a national 
achievement examination, a national exam, with the same ques- 
tion asked in California and Connecticut. I don't know why we 
can't do this. Photosynthesis is the same in California as it is in 
Connecticut, and the quadratic equation is the same in any one of 
our States, it doesn't change. It should be an achievement exam, 
not an aptitude exam. It should linked to the curriculum so ev- 
eryone knows what is expected. The objective i — the teachers would 
teach in a variety of ways to meet these objectives, and the results 
would be known. For the first time in our history, we would have 
goals and accountability; results will matter, and attention will be 
paid. 

Thank you. 

(The prepared statement of Dr. Saul Cooperman follows:) 
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Mr. Chairman, members of the Coaaittee, I am SauI Cooporman, 
Vresident of Educate America , and I am pleased to bo here this 
morning on behalf of former New Jersey Governor Thomas M. Kean. 
1 thank you for this opportunity to present Educate America's 
propo&al . 

Educate America believes we should have a required National 
Achievement Examination for all high school seniors. Such an 
examination would provide a reliable accountability system to 
students, schools and states. 

Educate America believes that wc ought to have a core 
curriculum; we ought to state what children should know in key 
subject a -eas and determine whether they know it or not. If 
they don't, we £5hould challenge the schools who don't measure 
up, and leatn from those who do. Make the ends drive the means, 
make rer,ults drive the system. 

Let me give you a specific example of the present system and 
how fducate America's idea for a National Achievement Kxam will 
change things for he better* 

In every hi-^h school, students who do not plan to attend 
collcqie know that the diploma counts, but not much else 
natters. These students know employers will ask '*do you have a 
dipioma'' so it is important to thcw. But, during the last 
twenty-five years the ••general track" has increased dramatically 
as more and more students rake general science and general math 
rather than Biology and Algebra. v:ny not, they reason, all arc 
worth five credits. 

We have given the message to students that taking tough 
subjects doesn't count and that hard work doesn't count. 
Educate America believes we've got to get our educational system 
back on track. We've got to deliver the message that working 
hard means something; results mean something. We've got to link 
real accomplishment m school and the world of work. The 
meti^iage should be, "Do well in school and you'll get a bettor 
job, at more pay, than if you graduate with a mediocre record." 

Until now, there was no way to take these words and make 
thrm a reality. Now wc have a way. President Buiih and the 
nation's Governors have established national goals in reading, 
writing, math, science, history and geography. If we have the 
willpower to take the next logical step and ask what seniors 
must know in these areas, we can then create state of thv .irt 
ex.^ri n.*t ion to deterrine what students real know and ar-* ablo 
to (Jo. Ihon busi norir.poDpi e, who ar.X student (or their Sv.'orer. 
can have solid information upon which to reward accomplishment. 

A National Achievement Examination har, the power to chanrjr 
things by galvanizing the public. Results will be rloar and 
easily understood. Right now people don't pay much attention to 
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international coapdrioons or NAEP results. As sorae Hew Joreey 
educators told me a fow years ago, "Saul, I Know tho general 
picture might not look too good, but fijftt schools are oXay, so 
why should w© worry. Because the results wrere baaed on a 
sampling of students, the message never came to tho specific 
school, with a National Achievement Examination, results would 
be known for every high school in the country. Tho message of 
results, the message of facts would be publicly delivered to 
every high school door. And facts are stubborn things! 

For the first time in our history, we will have a National 
Achievement Examination, A national examination, with the same 
questions asked in California as in Connecticut; an fl£hiffV?n»?nt 
exam, not an aptitude test; an examination for alLBfialfiMr not 
a sampling of seniors » We will, for the first time in our 
history as a nation, have goals and accountability. Results 
will matter and attention will be paid. What gets measured, 
will get done. 
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Chairman Kildee. Thank you. 
Dr. Hertz. 

Dr. Hektz. Mr. Chairman and memb^'^ of the subcommittee, on 
behalf of the American Association of School Administrators and 
on my own behalf, I would like to thank you for the opportunity to 
testify on this mc^t important issue. 

I am superintendent of the Mequon-Thiensville School District in 
suburban Milwaukee, Wisconsin, and if Mr. Owens were here or if 
one of his staff memters is here, I would like to further establish 
credibility with him by saying that I was an English teacher when 
I was in the classroom. 

During the past year, I have served on the Wisconsin Governor's 
Commission on Schools for the 21st Century which flowed from the 
President's gathering of governors regarding education, and I 
chaired the part on assei^ment. 

AASA would like to address three issues surrounding the devel- 
opment of national tests. First of all, can a national test be the 
m^or force in improving education? The short answer is no. A na- 
tional test cannot be the major force in pr(Klucing educational qual- 
ity. First, a national test would hold students responsible for condi- 
tions they do not create nor do they control. 

An associated dramatic concern connected to national testing is 
the inappropriate placement of disadvantaged children, especially 
those who come from nonstandard English speaking homes and 
classes that do not challenge their true talents. 

Second, a national test would occur separately from the daily 
learning process. Because the test would be removed from the in- 
structional setting, the results are unlikely to be important to the 
students themselves. Further, the Wisconsin Commission's report 
identified r.he relationship between the assessment process and the 
immediate and direct application of that information back into the 
instructional process. 

Third, the notion that quality can be achieved chiefly through an 
end-of-the-process insf>ection is wrong. Quality cannot be inspected 
into a product; rather, quality can only be built in. Edwards 
Deming is all over this country and all over this world talking 
about that in all sorts of forums. 

Fourth, developing a single national test will take a long time, 
and, given the need to hold down expense, the test will likely have 
to be multiple cl.^ice or short answer tests. 

I would also suggest that the construction of a national test will 
be fraught with political struggles which will neutralize any poten- 
tial value it might hope to have. 

Fifth, a multiple choice, short answer test will almost certainly 
fail to accurately measure student learning, because a national test 
will not test what is being taught. 

Since there is no national consensus about what students need to 
know and be able to do, no national test has any basis for choosing 
what to test. 

The second AASA issue: Can test scores force educators to do a 
better job? Again, the short answer is no. Teachers and administra- 
tors whose students do not score well can be humiliated, but in the 
absence of clear information about the causes of failure to learn, 
improvements cannot be made. 
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AASA's third issue: Is there an alternative to a national t^t that 
can provide information about what students know and can, at the 
same time« be useful to both instructional proce^es and political 
leaders? 

The short answer is an emphatic y^. School districts that have 
made great strides using information describing student learning, 
mastery, and have traits in common — first, such school districts 
b^an by clearly describing what students are expected to know 
and be able to do when they finish school. Second^ the districts did 
a careful study of how tUry could best facilitate the desired out- 
come through structure and curriculum. Third, they spend a con- 
siderable time in teams deciding what evidence of pn^ress and 
mastery would l^t facilitate learning and improve teaching. Final- 
ly, the school districts carefully studied what more they needed to 
know and be able to teach every student the desired outcomes and 
use information about learning to continuously improve. 

There must be a consensus on what students are expected to 
know and be able to do, otherwise there is no basis for judging 
growth. And the fact is that different districts and different loca- 
tions and at different points in their history have varying expecta- 
tions, goals and aspirations. When a national trat is given in such a 
setting it can quickly become a ojmmon level of minimum compe- 
tency, which can cause huge numbers of students and other people 
to focus upon an expectation which is considerably out of sync with 
the condition of a given community. 

During the deliberations of the Wisconsin Commission on Schools 
for the 21st Century the Ass«^ment Committee heard over and 
over again that norm reference t^ts were not used to improve the 
instruction of children. When our district receives the results of na- 
tional testing or is doing assessment of writing or individual testing 
by teachers in the schools, we take the results into small groups of 
teachers and building administrators and plan exactly what the 
educational strategy will be to effectively use that information to 
better instruct the children. 

We feel that an immediate use — and I would like to underscore 
the immediate use of assessment information to feed back into the 
instruction and learning process is essential. Please let me put 
forth AASA*8 proposal to improve both the information used by 
teachers and information desired by policymakers. 

Rather than a national t^, we need a consensus on what stu- 
dents are expected to know and be able to do. The National Council 
of Teachers of Mathematics have developed a splendid set of stand- 
ards for mathematics that serve as an example, and I can say that 
in our school district we have just this year implemented a curricu- 
lum, K-12, using th(^ standards. The science, English, social stud- 
ies and reading teachers have and are doing similar work. 

Congress should capitalize on these efforts as well as the work of 
the National Assessment of Educational Progress and various aca- 
demic commissions to develop a national consensus on standards 
for education. And by standards, I am talking about expectations, 
not something that is a rigid level for all school districts. Because I 
don't think, personally, that you can come up with something that 
will fit every school district in this Nation. 
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We do not need a single list of desired outcomes, but rather the 
national standards framework called for by Lauren Resnick recent- 
ly in testimony to the Romer pane] on implementation of the na- 
tional goals. These standards are most definitely not a curriculum, 
and most emphatically not a test. Rather, they describe what stu- 
dents should know and he able to do. Statra and i^h(K>I districts can 
then determine how to achieve the standards and which standards 
should be emphasized in that particular setting. 

If employers could have accurate and easily understood descrip- 
tions of what students knew and could do, and if all students had 
achieved high standards, there would be little use for other test in- 
formation. I have i^rsonally told business people in our community 
that they do not use our quality control mechanism which has ex- 
isted for a long time; namely, grades. 

It is extremely unusual — and I suspect if you go back to your 
communities you will find the same thing if you ask the principals. 
It is extremely unusual for an employer to ask for a student's tran- 
script when he or she is being hired. This would tell the employer 
a great deal about the degree of success of the individual ana rein- 
force for students that their efforts in school do count. 

Now the reality is that when a businessman says to me, *iVe got 
a young woman from your high school, KarK and she can't tyj>e, ' I 
talk to them about this topic, and I say, "Do you know whether she 
was a D student in typing, or a C student, or a B student, or an A 
student?*' If she was an A or B student and she can't type, that is 
news to me. If she got a D-minus, we all knew; we knew it and you 
could know that she wasn't one of our good ones, you see. And so 
the reality is we don't use the mechanism that we nave for quality 
control of individuals as it is. 

In the past, I have spent a great deal of time listening, thinking 
and talking about this, particularly in the last year, especially the 
subject of assessment of children's progress. It is my feeling that 
there is much to be gained in the instructional arena from assess- 
ing the progress of children in a way which can be, again, under- 
lined for you, immediately fed back into the improvement of in- 
struction. 

However, I must say that I personally feel that s(K:iety has 
pinned entirely too much of its hopes lor the improvement of 
American education on the testing of children. In some way I sense 
that we hope that by having great national testing of children that 
we will embarrass ourselves into being loiter. 

Then there is the unanswered question in my mind of whether 
there is linkage on the national scene between these tests and 
other agendas such as choice. There are other facets of the educa 
tion of children which it seems to me are far more at the core of 
the improving of the learning of children such as (1 1 improving the 
health, family situation and learning of preschool children in disad- 
vantaged situations; i2) demanding an acceptance by all of the edu- 
cators that every child is capable of learning no matter what the 
youngster s background is and that we do not automatically catego- 
rize some students as being academically deficient because of their 
background, and that is happening: (3^ convincing ourselves as edu- 
cators that all of these children can be lifted to a high set of expec- 
tations and that we will demand the very best of them; and U> rev- 
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olutionizing parents' expectations that would lead them to accept 
as Asian parents do that hard work of children is the primary 
factor in their success in schwls. That is not what American par- 
ents are telling us. They are not telling us that hard work is at the 
core of their children being successful. They are telling us that it is 
what G(Hi hapi^ned to give their kids; in other words, they have 
either got it or they haven't got it, which in my mind takes every- 
Ixxly and the mechanism ofT the hook: the child is off the hook, the 
teacher is off the hook, the parents are ofT the hook; whereas, if 
you have the mind-set that hard work is going to do it, then we are 
all on the hook. And (5) making a national commitment that we 
would extend ourselves mentally and financially to improve educa- 
tion. 

On behalf of AASA, I thank you for the opportunity to express 
our views on national testing, assessment, and accountability. 
[The prepared statement of Dr. Karl V. Hertz follows:] 



ERIC ^ ' ^ 



215 




AMERICAN ASSOCMTION 
OF S(»OOL AOMlNiSnUIOAS 



TESTIMONY 



OF 

on. Kmi* V, MERT2 
SUPERIKTENDUrr 
MEQU0N-THIEN8V1UX SCHOOL DISTRICT 
MEQUOW, WISCCMiSTN 



KATTONAL TBSTlHG/ASSEfiSKBNT AND ACCWKTABTLITV 



SUBCOMMITTEE OK ELEMEJJTARV, SECONDARY, 
AND VOCATIOHAL EDUCATION 



UNITED STATES MOUSE OF REPRESENTATIVES 



THE HONORABLE DALE E. KtLDEE 
CHAIRMAN 

THURSDAY, MARCH 14, 1$$X 
WASKINOTONi DC 



ON 



TO THE 




216 



Mr- Oiairman ^nd aeobere of th* »MbcoBiiaitta*, on bahaXf of the 
teerican Association of School Adxainistratora CMSA) , i vouia liKa 
to thank you for tha opportunity to taatify on thia iDoat important 
isaua. AASA l» tha profaaaional organlEation of over 19,000 achool 
aiatrlot fiuparintanaanta, oth«r locol aOniniatrators, andt 
profasaora of adiucational adainistration. in our capacity as 
education CEO 'a in our conaotinitiaa, w« hav* a vital intar^at in tha 
discuBSiona ra9ardin9 poaaibla davelopnant of a national test. 

I am Buparintandant of the M«qwon-Thi»navilla School District in 
Mequon, wlaconain which ia a suburb of HilwauKaa. our Uit^trict 
haa an enroHment of 3,^oo atudants amployin? 240 profasalonal 
ataff. During the past yaar, I hava sarv«d on tha Kisconsin 
Sovarnor's ComnisBion on schoola for tha 21at Cantury which flowed 
from the president *s ^atharlng of govarnora regarding education. 
I chaired the work on asseaainent. 

AASA would like to addrasa thraa of tha issuaa surrounding tha 
davalopmant of a national taat« 

CAN A KAT IQWAL TEST i;^^ ffAJOR fQRCE IN IMPROVING EDUCATION? 

The short answer ia no, a national taat cannot be the roajor force 
in produizinf$ edacatiortAl quality. First/ a nAti.unal test wuuld 
hold students responsible for conditions they do n-^t create or 
control. In general, children and youth who live in poverty score 
worse on tests than economically advantaged atudanta* Children 
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whose are wracked by poverty* tanily tragefilas, and major 

personal probXexBs often do not do veil on teste unless the 
underlying causes are atnelioroted. Similarly, w» cannot say with 
certainty what eSSect pervasive racism and accompanying luw 
expectations hava on atudent performance, but it must l>e 
conaiderable. An aeeociated cSranatic concern connected to national 
testing is the inappropriate placement oi disadvantaged children, 
espacially those who come from non-standard English speaking homes, 
in cXassGB that do not challenge their true talents. 

Second, a national test would occur separately rrom the daily 
learning process. Tests have been aekad to fill two needs. 
Instructional information and public accountability. No current 
test Is able to fill both needs completely although aome states 
are working toward that end. Because the test would be removed 
from the instructions! 88tt.ing, th« rvsults arm unlikely to b« 
important to students. Experience in state testing shows that: 
tests which are viewed as unimportant do not elicit the best 
efforts of students. Another low interest factor for students is 
related to the fact that they are separatso in time from the testi 
further reducing the test's significancs to atuUenta. As lew 
Rhodes of tha aasa staff recont-ly wrote ^ "By removing assessment 
from any functional role in the i nstruct loneii process, we have 
virtually mai^n it iinposHiblf* for tesching to be a continuously 
improving process under ccintrol of a, ieach«»r? or more ir.port ant : y , 
for learning to be a continuously improving process under control 
of the learner." Further, the Wisconsin Commission's report 
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identified the relationship between the ae*e««ttant process and the 
Immediftte and direct application of that information back into the 
instructional process. To teach children l:^^tter n gv i» the number 
one purpose of asssssmsnt* If testing is not dirsctly and 
imaediately linked to improved instruction for children, it has 
little chance of havin9 iispact. 

Third, the notion that quality con be acnieved chiefly through an 
••end of the process inspect ion** is vrong. Quality cannot be 
inspected into a product? rather, quality can only be built in. 
Perhaps the committee should invite the father of total quality 
management, w. Edwards Oe»lng, to SKplain that principle and 
is-»rev«r put to rest the myth that any type of end of process 
inspection, including tests, viiX result in quality for any 
activity, Including education* 

v^ourtn, devsloping a single nationsl teet will take a long tioe, 
and given the need to hold down expense, the test win lixely have 
to be a multiple choice or shoxrt ansvsr test* Test maKsrs are 
worKing on improved norm referenced end criterion referenced 
standardised tests, but tests now rsly on multiple choice ur short 
{»nswer test items rather than deaoast rat ions of learning* waiting 
on a national test to solve educational problems, and then 
discovering that testing will not improve the syetein will taKe 
years, maybe decades* we simpi/ cannot take that much time. T 
would also suggest that the construction of a national tsst win 
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tm fraught with political »truggla« which will nautraXise any 
potMtial valua it might hope to have* 

Fifth, a aultipia choice snort answer tast will almoat cartainly 
fail to accurately measure student learning bacauut? a national toot 
will not tdst what la heing taught. Also, work on learning etyln 
and multiple intelX igencea ahovs clearly that how guestiona are 
asked haa a great deal to do with how well students perform. Ko 
single test can respond to tha learning styles found in any large 
group of students. Final ly> since there is no national consensus 
about what students need to know and be able to do, no national 
test has any basis for choosing what to test. 

CAN TEST SCORES FORCE EDUCATORS TQ DO A BETTER JQ^J 

Again, the short answer is no. Teachers and admin istratora whoee 
students do not score well can be humiliated, but in the absence 
of clear information about the causes of failure to learn, 
lajproveiuents cannot be made. The notion that humiiiation can 
produce iTwproved results assumes that teachers and admin istratora 
ate holding something bacX in their professional practice that they 
Know how to do better but for aome reason will not do« Lvery study 
of teachers and principals has found that fpost educators do their 
best each d^y. An asfiuisption that educators Xnow sonethincj they 
are net using has no supporting evidence. 

Teachers and adnmistrators roust ijnprov© their competence in 
developing asse&snjents and using assessment results to facilitate 
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student learning and improve their teaching* A national test 
developad by either the federal government or fione private group 
would not add to this pressing need* 

IS THERE AM ALTERNATIVE , TO A WATIOHAL TgST THAT CAK PROVIDE 
IMr pM^TIQM ABOUT W HAT STUDENTS KMOW AMD CAN AT THE SAME TXMH BE 
USEFUL TO BOTH THE IKSTRUCTION AL ^RQCESS ANU POLITICAL LEADERS? 
The eiiort answer is an emphatic yes. A number of states are now 
developing information Byeteas about student learning that can 
serve both the need for daily information on what has been learned 
and the need of political leaders to understand the impact of 
educational policies. Vensont, California, Connecticut, and 
Arizona come to mind as states worXlng on leorning information 
systems tnat can serve all masters. Sven more hopefully, several 
hundred school dlBtrlcts have been developing powerful information 
about student learning for use by students, teachers« parents, and 
administrators . 

school districts that have made great strides using information 
describing student learning and mastery hove several traits in 
common* Tirst, such school districts began by clearly describing 
what etudentK nre expected to Know and be able to wlittn they 
finish school. Second, the districts did a careful study or r.tjw 
tney could bast facilitate the desired outcome tr.rough slruuturc 
and curt^culuiT. Third, they spent consideraoio time in leans 
deciding what evidence of progress and mastery would best 
facilitate learning and improve teaching. Finally, the school 
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districts CAr^fully 8tudl«dl i^h&t mor* tttmy nmmdmd to Know &n4 m 
abXo to tfo to teach every etudent %M dMlr^d outcomss and Mmm 
infoniAtion about l«arnin9 to continuously ifoprove. 

School districts en^a^ed in tundasentai cnan9e have learned that 
Changs takss tiss and that tnsy must strivs for constant 
iiBprovsmsnt * Thsrs ars no quicx answsrs in sducation or any 
cosspisx human activity. Ths challsn^s is to accspt th« principle 
of continuous inprovsasnt rathsr than try Qassivs instant Changs. 

Ths transformation depends upon InforMtion about student isarning 
and behavior that first and forsmost facilitates student progress 
and ths professional practices of c^ducators. Howsvsr^ aggrsgats 
infon&stion on studsnts* gains can )>s described in t-srms of tns 
locally developed outcomss# ststs-wids tests^ or commsrcialXy 
availabls tssts* 

Thsrs srs many types of information about student progress, some 
of which csn be aggregated upward and ussd throughout a syatea in 
ths sort of examination systss snvisioned by Laursn RssnicK* But 
first, there must bs a conssnsus on what studsnts are svpected to 
know and be able to do, otherwise t.hsrs is no basis for Judging 
growth. Only after sKpoctations have bssn dsfined is it rsasonabls 
to ask what evidence ie &ccsptabls that learning has taken place. 

The fact is that different districts in different locations and at 
diffsrsnt points in history have varying expectations, goals, and 
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mh ^aieUy M«dm a liral of jilniMim eoa^titrnm^ imioh ran 

jMUM tiuq^ tmiter* ^ p«^n to focnps upra na «xpfKiftntim «hiGli In 
imyiidnrnMy <nit nf nith tUn MadltAm in n «iv«n nmBmnl^a 

Owing tlm tfniibnMtinnn.of cte IVinMMin ConinnicMH nn sobooln (or. 

tJut mm ncnmnraa tnstn vmrm not UMd to Inprovo tho inntruetioa 
chtl^fTOr xt toonnn oiMf to m tlmt ju^iny tho oiiccot> o( 
omidimn rauid to dmn in n vnrioty of ciMrly^ hv leoldn^ 

dnt tho witiag^ portfoiiM^ ovporiM^Ov ond proioeto of omidrwiit 
tOMtoro mxm particminriy «l>i« to tote vhnt ttoy tovo ionrrad otottt 
tto otiMMmo* pxoffvnm MM UmA it dixwtly took into tlio- 
iootruotionoi pzoerao vittiin • vory otort poriod of tino idaoii io 
ooMntinl to aaJOnv tHo looming offootivo. iliio oon ootmiiy to 
dono with nom roforoneirf tooto too* Bwovor, tto timing io onoo 
again doloyod to o cortoin ontont, ondir of omroo« in ony Itind of 
notiorai toot it Mould to doloyod dnaotiool^y ond o^rot^ fron 
tto Imrning f^nemwm to on ovon gnotor d^r^roo* lOion our diotriot 
roooivoo tto rMulto of notionot tooting^ or io doing oooooonMt 
of vriting or any of tto othw noro iswiioto forao of aooooonont, 
%io toto tho roottlto into onoil grwpo of toootoro and building 
odminiatratoro and plan onaetly tfhat tto odoeational otrotogy will 
to to offootivoly uoo ttot infomation to tottor inotruet tto 
ehildran. In aoso eaaoOr thio nay roault in ohangoo wnioh vili 
toto plaeo inoadiatoly, in ottor inataneao it nay Man ttot tto 
topic will to approaetod difforontly in anottor yo»r« and in oono 
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CAMS thm tM» of •Oucators will d^cids tnat t.iie ohildrvn (Sid 
poorly tout wm did not intend to tMch that topic, and do we try to 
avoid ft precipitous raaction to try to oddraaa a situation that tra 
aay Know ia cov«rad vary appropriataly at aom otnar point in tha 
child's Xaarnirvg situation. Wa fssl that an immadiats usa of the 
aasasasant information to faad back into the instruction and 
laaminq procaas is aaasntial. 

pipaaa lat ma out forth AASX*a prooQsaX to improva fcM?th tho 
inforaatton uasd by f achara and information daaifd bv pqIIcv 
makara > Rather than a national taat, ws nsad a consonsus on what 
atudanta ars axpsctad to Know and ba abl« to do. wa n««d a oassiva 
coraiaitttant to inprovad practicas ragarding student asaessmant so 
that tha question of accoptabls svidanca can ba anawered to the 
satiafaction of buainssa paopls, parants* sducators, and tha 
comounity at larga* 

The Hationul Council of Taachsrs of HathSMtics has d«v«lu|»*d a 
aplandid sat of standards for mathasatica that sarva as an 
excallent sxarapla* Tha acianos, Bn^liah, aocial studiss^ and 
raadin9 teachers have or are doing nlRtilar ^ork. Coni|reas sht^ulU 
capitalise on these efforts as well as the work of the national 
Assessmont of Educational Progress and various academic commissions 
to davslop a national consanaus on standards for education. 
Becauso all of the work on standards has bsan dona in amall slits 
professional groups^ a national consensus on ths important navt 
4tmp haa aXudad us. 

s 
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Con^r^M •noultf act quickly to afttaJOXish a Droatsly baaed coalition 
to aehiava tna naceasary consanaua on what we expect atudanta to 
know and ba abla to do* Thia ahould ba dona in conjunction with 
the Romar panal, but without repoating tha mlstaKe of only talking 
with a narrowband of individuala. Ratha< , tha congteaaional panal 
ahould apand one or two yeara achiavin? the consenaua on atandatda 
by building on tha work already done, being intluaiva, and by 
actually listaning and achieving conaanaua on what parents, 
business, higher education, and tha public at large actually want. 

Wa do not need a single list of desired out coses, but rather tha 
**national standards framework*^ called for by Lauren HasnicK 
racantiy in tastimony to tha J^ower panel on implententat ion of the 
national goals* Theaa standards ara sost daf in Italy not s 
curriculuTD «nd &ost emphatically not a teat. Kather, they 
described what students should know and be able to do. states and 
achoo?. districts can than datart&ina how to achieve the atandarda 
and which standards should ba amphoalzed. After there Is a 
conaenaua on expectations, the guest lone are, first, how do we 
achiava those expectations as soon as possible? Than, what 
avidanc* of at aidant prograaa and maatary ara required to facilitate 
continuous learning and improve educational practice and structure? 
Finally, the queation ot how to aaaass tha evidence of performance 
success ahould be answered. 

If eaployera could nave accurate and easily understood descriptions 
of what students knew and could do ^nd il all students had achieved 
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hi^h •tandttrtB, thsr* wouXtf b« littic ub* for other 
inSora^tlon. I tuive porftonally told tou»in«»» p^opls that tb«y 
not UB» our quality control aechanira which Mist* - naoely gradM. 
It Is •ictr«ia*ly unusual for an enployar to aaX for a ■tudant*^ 
transcript whan ha or aha ie baing hirad. Thia would tall the 
amployar a graat daal about the dagraa of auccees of tha individual 
and reinforca for atudents that thair efforts in school do count. 

KoKtf congrasa must approprlata funds to support the developaant 
of coB^etanca in studant assesBmanta and in using that Information 
with atudants. studenta uaa information in all othar aspect* of 
thdir livaa to maka decisions; thay can use assessiiiant information 
ju9t as wall. 

In tha paat year, I have spent a great deal of ti»e lietening* 
thinking, and talking rag^4rding the subject of aa^aaament uf 
childran'a progress. It ia my feeling that there ia much to be 
gained in the tnatructional arena from asaeasing the progress of 
chliaran in a way which can be inuBediately fed back into the 
iwptovecent of instruction. However. I must say that I personally 
feel that aoclety has pinned entirely too much of ita hopaa for the 
improvainant of American education on the testing or children, in 
some way I sensci that we hope that by having gre^t national teating 
of Children that we will eabarrasa ourselves into being better. 
Then there is the unanswered question of whether there is linkage 
on tha national acana between thaaa testa and othar agenda auch as 
choice. 
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Tli#r« «r» other fac«t« ot th« eauoation of childrAn which it mmmmm 
to mm are fer »Qr« et the core of iatproving the learning ot 
Children euch est one, improving the health, family altuation, and 
learning of pre-school children in diaodvantaqed »ituationei two, 
demandinQ an acceptance by all of our eduvatoia that every child 
ie capable of learning no matter »fhat the youngeter's background 
Is end that we do not automatically oat^orice som etudente ae 
being academically deficient bocauee of their background; three « 
convincing ourselves as educators that all of these childxen can 
be lifted to a hi^h set of expectatlone and that we will daoand the 
very beat from tnet&; four, revolutionising p^arente* expectotione 
that would lead them to accept, aa Asian paranta do, that the hard 
work of children la the primary factor in their succeee in achoo. 
and, five, making a national commitment that we would extend 
ourselvec mentally and financially to improve education. 

On behalf of AASA, 1 thank you for the opportunity to expreaa our 
viewc on national teeting, aaee*sment« and accountability. 
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Chairman Kildee. Thank you. Dr. Hertz. 
Dr. Neili? 

Dr. Neiix. Thank vou, Mr. Chairman, members of the panel. 
Thank you very much for inviting FairTest to present testimony 
here tooay. My name is Monty NeiTl I am the ^sociate Director of 
FairTest, which is the Nation's only organization whose sole pur- 
pc^e is to look at and try to ensure that the ass^sment of our Na- 
tion's students and workers is made fair, open, relevant, and 
indeed, helpful. I would like to summarize my written testimony. 

Ba^ on our examination of existing proposals, FairT^t urges 
the House to support education reform by not implementing a na- 
tional examination or examination system at this time. The House 
should, however, support efforts to intr(Kluce new assessment meth- 
ods as part of improving school quality. 

National testing proposals assume that more measurement will 

groduce positive change. Recent history shows this is not true, 
luring the 1980s U.S. school children became the most overtested 
students in the world, but the dej^ired improvements did not occur. 
Adding more testing is clearly not the way to improve education 
anymore than taking the temperature of a patient more often will 
reauce his or her fever. 

Successful educational reform must begin by defining the kind of 
education we want our children to have. Not simply standards, but 
what experiences do we want our children to live through every 
day when they go to schc^l. On that basis, we can then determine 
the changes in standards, curriculum, school governance, and as- 
sessment required to reach the educational goals, and on that basis 
we can then decide whether we want or need a national assessment 
system. 

Current proposals for a national test or examination system seek 
to test before the nece^ary decisions atout the goals and methods 
of school reform have been made. This is putting the cart before 
the horse. Some current propc^als, such as the one presented by 
Mr. Coopciman from Educate America, call for the creation of a 
test that would cost about $30 for six tests, to be administered to 
all students sometime in the near future. Because of cost and time 
factors, such a test inevitably wilt be entirely or mc^Iy multiple 
choice. To do a performanw-based test, let me give you an example. 
The ART Advanced Placement Test cost between $60 and $70 a 
year. If you multiply that tim^ 6 tim^ 3 million students, we are 
spending on the order of a billion dollars a year to have a test com- 
parable in quality to the ART AP Test, not any $80 per student. 

In my written testimony I have explained why the imposition of 
a national multiplechoice or short-answer-type test will be an edu* 
cational disaster for the country. It is because of their limitations 
as technical instruments and because of their very harmful effects 
on curriculum and instruction. Yesterday, Representative Goodling 
read into the record a letter from the Pennsylvania Secretary of 
Education that I wish I had written myself, because it summarized 
extremely well the drawbacks of that kind of testing. 

No proposal that relies predominantly or very much on multiple 
choice testing should be given any serious a>nsideration by the 
House. Because the National Assessment of Educationad Pn»ress» 
about which you heard yesterday, remains predominantly multiple 
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choice, any effort to transform it into a national test will have 
similar disastrous effects on educational reform. NAEP should 
remain what is, a national indicator, but should move toward more 
performance-based a^essment methods. 

Now what about implementing a national performance-based ex- 
amination system, which is really the primary discussion that went 
on in the previous panel. It certainly has the advantage over multi- 
pIeK!hoice testing in that it uses methods that can assess high order 
thinking abilities and that sup{X)rt good educational practices that 
in turn could develop the creativity, problem-solving abilities, rea- 
soning, cooperative work habits, and so forth, of the students that I 
think we want to ittain. 

However, it is 3ur telief that we can move toward the national 
use of such assessments without constructing a national examina- 
tion system. We .'^ouM then gain the advantages of good assess- 
ment and avoid the ciisad vantages of impeding a national examina- 
tion system. 

Testing by itself of any type, getting information by itself of any 
type in itself will not produce reform. Information has to be used 
in the context, a much broader of — that is, outcome information 
from students has to be put in a much broader context of knowing 
about the inputs, the social factors around the school, the processes 
that go on in the school, and indeed other output measures. So, if 
we focus only on toting and assMsment as a solution to our educa- 
tional problems, we will end up failing to address such critical 
issues as equity, rigid and bureaucratic schcN)! governance and 
structure, low quality textbooks, inadequate schools of education, a 
lack of information about everything else we need to know about. 

We would be also imposing on teachers a new examination and 
the teachers will not be able at this time to use them. To make real 
use of performance-based assessments requires creating perform- 
ance-oriented schools, which in turn requires restructuring, staff 
development, and new educational materials. In short, we would be 
ill-advised to jump first to a national test tefore figuring out the 
other things that we need to do. We must oi^anize the pieces of a 
program of school reform into a coherent whole, rather than jump 
at one part. 

We also don*t know at this time whether it is feasible to con- 
struct a national examination system of the sort described earlier. 
The whole progress could prove too complex, expensive, and un- 
wieldy to work. Britain, for example, my understanding is, has just 
dropped moderation from its national system. Now, moderation is a 
process whereby the teachers grade the examinations but they do 
so in a process of constant discussion among each other in order to 
constantly redefine what is good work as distinguished from aver- 
age workt as distinguished from unacceptable work. It is also a 
proce^ that feeds back into redesigning the assessments. It is a 
process whereby teachers learn a lot. It is also a process by which 
the curriculum can be restructured. It is, in short, crucial to a per- 
formance-based examination system. But, as I said, Britain dropped 
it, largely because it is too expensive. Britain does not test remote- 
ly the number of students that we were talking about testing. 

FairTest also is very concerned about issues of bias and fallibili- 
ty. On any examination, there will be students who fail who should 
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have passed, and experience tells us, from the history of examina- 
tions in this country, that a student from low-income and minority 
group backgrounds will be disproportionately among the students 
who fail the tests but should have passed the tests, and who there- 
fore suffer unfair negative consequence from testing. 

While I am sure that the proponents of a national examination 
system do not support tracking or grade retention, we have no par- 
ticular reason to TOlieve that by setting up a situation whereby stu- 
dents are expected to attain a certain level of mastery by the age of 
16 that it will not in turn feed back to a proce^ of tracking and 
freouent retention. Indeed, the discussion earlier su^ested that 
students who don't attain a certain level at grade 4 might be re- 
tained. We know about as well as we know anything about educa- 
tion that retention in grade simply does not work. 

I like the idea of more time in order to accomplish what is 
needed, but it can't be done by retention. The message then is we 
have to think about rretructuring things like grade levels and 
things like use of time in the school day. We have experiments 
around the country on how to do that. We need to learn a lot 
before we drive everything with a national examination. 

Instead of implementing a national exam at this time or a set 
date such as the year 2000 by which we can expect such an exami- 
nation system to be in place, FairTest urg^ tne Federal Govern- 
ment to take the following steps to improve educational assess- 
ment. 

First, assist States and districts acting in consortia to develop 
and implement performance-based methods of asse^raent. For Ex- 
ample, the various consortia working on new assessments such as 
the Learning Resource and Ifevelopment Center at Pittsburgh and 
the National Center for Eklucation and the Economy, as well as the 
consortia from the Chief State School Officers, another one from 
the Education Commission of the States, another one out of Boston 
College headed by George Madaus from the National Commission 
on Testing and Public Policy—they have efforts under way. Their 
efforts should be supported. This means funding research and pilot 
program implementation. 

You should also assist State and district teacher education and 
staff development programs because if t^hers are not able to use 
the new assessments, the new a^e^ments will not help us improve 
education. 

You could assist the subject area group® to develop and dissemi- 
nate model frameworks, curricula, standards, and assessments. 

Also, assist all of these diflerent levels of gro ups in sharing infor- 
mation among themselves, such as the NuTiA standards, now do 
they get widely distributed around the country and how do teach- 
ers and administrators learn to use them? 

The standards we are talking about, I would add, are ultimately 
what the Coalition for Central Schools refer to as habits of mind, 
and what a group I am now working with has b^g:un to call intel- 
lectual competencies. It is not simply bits of information that 
people need to know, nor are they fixed things you have to be able 
to do at a certain time. Rather, it is, for example, our history stu- 
dents should learn to think the way historians think. Now there is 
a lot of levels at which that can be done and we should, indeed, set 
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levels that our students are shooting for and doesn't prohibit much 
higher levels such as that attained by professional historians. But 
that is the approach we need to take that is a radically different 
approach than most of our schools in this country use and it 
cannot be measured with most of the current testing instruments 
that we have. 

We also have to be sure that our ass^ment plans are embedded 
in thorough educational reform, and we urge you not to act unless 
you know they are. Don*t just assume that they will be. 

I would also urge you to reconsider how the Federal Government 
now mandates standardized testing such as with Chapter 1. All 
across the country I hear repeatedly what a disaster those man- 
dates are for good education in Chapter 1 programs. 

In conclusion, let us say we are not aiding at FairTest against 
accountability or for slowing down school reform. Rather, the cen- 
tral issue is how do we define education? How does our Nation 
define education? We n^ school reform, not more testing. Our 
Nation must not be misled into thinking that more testing will 
solve our educational problems. Indeed, good assessment is part of 
the solution, it is not the first step toward a solution. We will be 
far better served to take the time to do the job of school reform 
carefiilly and well than to a^^t hastily and poorly with destructive 
results. 

Thank you very much. 

[The prepared statement of Dr. Monty Neill follows:] 
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Wif tht Vntlfd States Uw* Not Need a Nstional Te&tt 
TcMimcmv to Ihr HtMise Comnnttee on E^tneiit»ry and Secondary Educvlkm 

by 

Moniy NciH. W D . Assoc laic DimHrf , 
Ndfimui Onici ftw hair & Open Testing (f aifTcM), Cambndgc. Mass 
Manh 14. I99I 

i.»d$es ^hJ lieniiCTwn. 

Th mi you v^ry mwh fm invjiing FairTcs! to appear at ihis important heanng 

Based on an examination of cxi»iing pntposals, ^auTrii contludcs that most cuTrent 
effons io eviahlivh a natuiiiaJ lesi lo measure progrtis toward the nation's educauonal gtia!\ 
wiU hurt, nm help, our nauon's efforts to improve school quaJjiy We therrfore urge the 
Hou^e of Representatives to support education refmm by n^f implemcnbng a natiooal exam si 
ih« time The House should, hoA-ever. lupporr efforts to introduce ne^ ai%cssment meth<ids 
as part of implementing ithool reform 

Ntfuona) testing proposals lar^ly are based cm the faJsc pnemtse that measurement by 
Itself will produce pOi»i>ve change Recent history shows this is not mie Ihmng, the 1980*. 
t*.S Khoo\ children became pTo^aWy the most over-tested students m the w.jrfd hm the 
desired ediK auonal improvements did not occur FairTest research mdtcaic . ' -^t our Si h4n»i*. 
now give mm than 200 miUton standardised exams each year and the typita} student must 
lAke several duten before graduaong * Adding mofe testing will nu more improve edocatKW 
than talmg the temperafuft of a patfcot mt^ne often will reduce his or her fever 

In conirasi, successful edi>rarK>naI jvUwm must include restnictunng curnculum. 
msmjction. textbooks and other matrnals. schod governance, and teacher eduiatufn. a^ well 
as asscsstneni What ^ need to c^te arc schc^is as communincs of and for learning 

To move towaid thai goal. itaihet>. admimstrator^, mhcf school personnel, paienis. 
stttdrnts. community members, and goverrm^em muft an he involved m an open and 
dermxrji*i prticess of defmmg owr educational goais ■■ at the tacal. state and national leseJs 

so that we can agn«. for example, on whst h means for aU students to be cc»npctcnt in 
diffctrnt areas On that hasis. we can then determme ho* to make the changes irqurred to 
reach the gtwls. including a decision on whether to institute a nauonaJ icst l«osi cufrrni 
pmpoiialv for a national le . however, seek to lest hef^trt necessary deciSKws about th^ goaK 
of school rfftirm have been made This hkely v-jil lead to the backdoor tniposition of a 
r>a!)raul turruulum. without pubbc discussion 

Indeed, havjwf a smgle nati£wal lest raises the issue of *e control of education If the 
lest becomes important, as ait tesung pmponems want, those who control the tea coukJ 
contr»)l curnculum and m$tnin*on. parucularly J decisions about curnculum tr i* mstnictton 
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Tg>Hmi>nv i^i Monty \f)t! <vfi N^tH^njI 1c\tmi! twuc^ 2 

h'd\'C m>t been ^nvcd at bcUnc ihc ir%t is consiruttrd. and maybe c\cn li thovr drtiisums 
havr hcen iravhed 

A natHtn^tl c^ani shtniW mn be allowed lo umlfmnw such fwrdrd a/ui enwr^inp 
frfomu as whinil havrd managrmcm and shdtrd dfciMon making By iTntrah/mg decJMon 
iTukmp. vcniralj/fd njiumal lotmg mmt Ukc)y *mH make cdu<. afjon no! morr, 
accountdble to pyre nix, studcmi. ic*itJ»cr\ ami ihr t4>mmun?i> II ihc tcM ccntijUy 
trcmtTiilkd. w vkhi\m a^uld pjrtnfv. Jcathm and crKnmunitir^ ap^^j) \( thc> di\appm^c o< ihc 
cwmcuiar dck JNUMj\ and invtnutMwal .nc!ht»d«» imjHisrd tHriHi|;h thf test * 

Aj>> njJu»na! tc^l or cxaniinaiMm sysicni iould pr^-dittic jdttifumjl major prohicins 
I i>r c^an^plc. the examination vvMcm\ nf many <«h?r natums arr h4«.cd m S4. h()(il vyMcnis thai 
ngidl) Mirt ihcjr >UKirn{\ ami prrpetuaif vtvial c!aN\ divi>KKi>. prrtJudrng the «Kial mnhihiv 
our natMHi hc!ie>c\ »n No tviic le^t Nhiniid hciomr a natnmal gdtei^eepef thai pcrpeMiaiCN nut 
naiH»n'^ itnfi>rtunate hixlorv- uf urdaifiy wriing \tudem\ hy fate andctavv 

A natJivMl !c\f uuiid end up hemg used to dctcrn>ine high h H^xiI graduaium. 
eniphnnient arvtJ cntrarke mio h»ghcf ediKainm I>ttc M> unav<»jdjhtr mravuremcnt rmc ami 
t'la*'. nuns siuitntx ^hi? fail j icnI ^mU, in re»iht>. be a\ capable av man\ wh^j paw Reveari h 
ifuiuajev ih.ti iht»*.e uhn fail hut sh4»uM ha^e pavW uill he di«kpn»p«irti<>n3tcU fn»ni hm 
ini*»mc and mjnorju ^ytmp hjkk^rtni»id> l ajflcNt ii^c^ with the Nat«»nai Commissum <in 
le*»tmj: an^t !*uMu Fot;^\ thai, hckau^c ot ilie bias and emtr. n^^ tine ^eM vh^^utd c\fr Ik {He 
voir ju pf»r7i.ir\ hjvjv makjng an impon^int cduvafuwai dciisjon ' 

Ujn^*et> ot Muhiple ChotkC lestmj: 

i j»?Ir-( rrv»>i:«tt/rv ihoi itirre are twi' ditfcnrm l\pestif prnp<»v.ds inr natjonai tcMin^' 
{h\c i\pc uiH e^^rmuiU iwi muliiplc iht»ue tesfm^. the tnhei vaiK ti»f p^'rtonnauve \i^\t A 
;»\\e\\mem Thee tvi(» appr,uvhe< ,i/c t|u{te dtflerem They are itie ddtctrme between 
fe^Itnj: *\tt.ii '.tuJt niK \h.'i4.'J Jin.-n and h/j»^; \ittJi'M\ in.*** i.^ * 

Ifu* ttfvj apprtuvh qtiukK Icj^K u\ niuhipic iHiM*.r tcvMn^.* of arhiirjry f^wN am! 
i^<il.ttc"t{ vkilK. un*.t»tu»ct(cd UMhe Hd> Km»*ledi:e i\ uved in the wtirld Multiple ch<»ue ^nd 
vhitrt aiivvkiM ifvi*. kanm»l adctjcialriy aN^es*. pf»»blem *.i»l\mg <>r the ahrljt) to cnr.tte and u^c 
kmiAledfp • lbj:hei «»rdrr thinking: rt^ojifv ihe vJudcJM to dcfme the pfuhlem, lotjmMdci 
an<f atiempi var?owv vihf?>onv to problcftix nhuh iirc itt >trut.turcd and may have tm^rr ihjii 
K^c turTi v! viOu!H>n. to prt^fLikC Kmmledj:e noi niereK fev4»j;m/e afisvser\ 

Hi'i jiive nndijple ih.«uc^h<m anw^er ic^ung ianm>l d«ekil\ a^vrss h{^hr« iwdi-r 
v.ip.ihjjiitev. a tt^t t.*»mprt\rt1 of suvh iiemv ndl mM mtttrrn un a\ tt* the prtihtem vtl\ tnp ^t? 1 
kmmk d^'f cuMtinj: i ^p.jbihupv ot tmi vitidensv We kfum Itohi v»f.!fwh. houevcf. th.»t 
\fudi nf jhittrtrv in thevr areas are very limucj '\h\\ has been iau^cd t trgcly bctau^c- sth«H»is 
of wh4K«K* failure to fea^h thi'm m 3n\ \uhtcst area to more than a few siLidrnt\ I \en ihc 
N si hi^'h svhmtl vtuilent^ upualK d<» m»i ktfov^ him fo pfithlem v»)\e (ujng the appr«»,Kh<*v 
ativ! n;rfht\I\ a pttifcwtonal u\e\ * Ves re^eaf^h aKo \ho>A v \hj\ pr<«Mffn \s'Kin|.«. knimjc;5^*i' 
vfratm'.* ,*ppT<uvhe^ i.»n be \^^\h \cf\ \oung thildfcfi ' 

U A tesl is H"p«»fUFa js a naU KiaJ tesi js ^urr to he thvn teachers wjfj fc.uh ?«' n 
HrK^UNC nu;!np?e ',h^>uc tc^is vannot dja<.tW nua^uic hi|:htf 4»Tde{ \kuU. Iea4.h;f4* M* the tesj 
fcdiKev rhnjinatex mstriKtioM tjmc spent oti the hif.'hef Instnutinn i\ reduved 

dnljm^' !i»f nujh»ple ihoue exjnis and the currttulum ts teduved tt> the te^? Multiple th<'ui- 
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TtMimonv of Mcmiv Nfii) on Nauonai Tcstma hsug^ 

tSKtmg precludes « cumculum based on ihtnkmg. tnve$$}gasing, pmhlrm M^vmg and ufung 
creativity, bccau^ rtrc tc« canwu measure those things 

Additionally, these cesis pcrpetoate the false tdea that ftrst stmienu learn ba^tc skills, 
then ihey leam higher skills. Cognitive ps^chc^ogical research has demonstrated thai Icammg 
mvtilvcs acfivc thinking and to enhance Icaming the stiident must he actively eogagrd ' 
Tcst-dnvcn schools produce higher test sews, but doj students who are able to think 

A pre.; tnunanliy muluplc choice test may iiKludr a wriiu>g sample A typical shon 
writing sample requires a vtudeni to wnfe several hundred words an a topic he or she may or 
may mit kntm anythmg ah*>ut and may or may not care about, m a shon pcnod of ume, with 
no charwe for rei^orch, discussion (thai is called cheating K or senous revision, fc^ no purpose 
dctpt the rest If the purpose of wi^hng is to commumcaie, then a typical test «'nttng 
sample cannot legitim3!cly be ca.led wniing at all. As wifh multiplc-chtHce wscng, it sends 
the wn>ng message about the goals of educaticm 

These tesis are nm very useful to teachers or policymakers The itastm, in both cases. 
IS that ihc rest rr^hv di> not help the tcm:hcr or policymaker decide wh<*i k* do next H 
Johnny ianmM muUiply, Ihc lew cannoi explain why If Mana's whole cUns cannoi n>uUipIy. 
the feM dtK-s n;>t ptmidc infomiation on what should be dofW 

Whai srandardired multiple-choice rests do best it help son students the go<x5, the hail 
and the ugly It is wh.>t they were invented to do But if we are verwus about refonni»t|5 
education so that all students can learn the things we deem imporiant. then we musx sti»p 
relying on ic%ls lhat has-e as their only real use the sowing of students 

In sum. tmplemenung a naiicviai roultiplf -choice citam will nmkad the public atH»u! 
the nature of the problem and iJje requirements of real change, block p^itive school ref<»rm 
ftnituding the ave of new methods of asN^smenl). htnder studrni*' ability to dcvekip the 
kinds of intellectual ctunpetencies they need, to develop, and ultimately undermine publk 
educatkm 

No pri^posal that suggests usjng mwe than s small pftiptmum of mulnple choice item^ 
in s national eKammatitin should he given any scrums consjdcration by the House At most, 
muhiple chOice could be used as pan of a sampling program to gather limited mformatKw 
aNmt student aiquiMlKin of a narrow range of knowledge There is no reasiwi lo test every 
student Uv this purpose and such a purpose should ncsTr he allow-ed to dominate education, 
a^ « to<» often now dcies 

Because of current technical hmttasions. any proposal to assess our nation's students 
inespensis-ely and in ihe near future will, of necessily. be a muhiple choice rest An esampic 
ts ihe pr<^posal by Ediicafe Amenca m test all high schcxiJ senK>r> m six subjects for $M> 
$50 each 5uth pn>p<>s3ls must be rejected 



Bv contrast, students should be assessed on whu) they Know how todt* To km»w how 
lodo sftmethmg mtludrs knowing factual tontcnt This methcid of assessment Cfurre^pe^nds to 
hitw people learn Ihev learn by integfanng new information <ir c^penensCs mt,-* the 
mif llectual framew*iTks ihey already pcnsess, whuh in turn enaMes them t<i refine and 
imprtn c the frameworks 
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many ways fur Mudf niv i« drmansnaie imeit«.iual tttmpc temc m and acfms fhi* suhi^i f 
arras Pcrfofmante based as5«4mrnti can he hascd cm fc|;u*jr MmJcnt iiassr<H'»m wurk 
prn^ccif* ir*cmh« wniingv, preiducr*, vclf nrnrctKw* tescttCf evaluation, cxhihmon^. and 
prrfcwmanrcv \hat can be wgam/rd and summan/ed m ponfohos In fum, Jh? f>f>rtfu]Mi\ 
can t>e examined by cnitside people - tcathm^ olhet purrnis, trained examimfrs to 
determine the quality o\ the ponfohm and the kimi* <if H-iTrk smdents air di>tn|j. Vermimi fm 
example. i% »«iking on ihi> mclhod 

Prrfiw7>ance- based assessments can also be examimiiums adminjiienrd fmm outride 
the cla*snH>m T^ese can include c^n-ended. complex priTbkrms teqmnng the Mudcn! u> 
figure out »hai «> do. solve the pn)bJem>, and explain what he or she djd Or chey can be 
e«hih»t;o«ii. performances and products, m'b as no^ done m Kience fairs. Scout Merit 
Badges, Advanced Placement An, and many performing and applied arts These ofien can be 
exams that are ^wmh teaching lo, unlike mubple-choice tests Anrona, Cahfomis. 
Ctwnecucut and Miin land arc among the Ntates implementing ihese types of exams 

Taken tt>gethet. m cbss and exifmally-developed performance based cxsm^ can 
encourage real work, model hjgh wandanls, spur jmpros-ements m teaching and cumculum. 
PfixSuce ins true tjonally useful inf^>rmation ftw tcachrr^ and students, and provide infonnafion 
based on real activiues about stuiJeni pn»gTess Avjie^smeni can play an imponam pait in 
dcvek^pmg cxHnniuni'les of and Urr leammg 

Cauiums on a Naiumal Performance Based t ^ am mat um System 
However, supp(wt for performance based assessments d >es mean sxKh assessment 
shi>tild immediately be transformed into a rational examination sv^iem, *u£h as that proposed 
by the l^arnmg Research Developmem Center and the National Center for F-ducstusn ami the 
tc<*nomy (l.RlK7\Ct.F) * Tliere are man); feasi»ns *hy ihts is the case Among ihem arc 
" Wr have rwit yet completed the process of discussing and debating nhat we ^jnt 
our educations] ?y>tem» to be ^Jany complex is&ues of educaifonal refimn* mvoKmg 
cumcular gi>als and standards, mstmctionai methods, assessment methods, school structure 
and governance, ami coJIectit>n of information, largely most he resolved hfftpr^ the quesiton «tf 
whether a national exammaiion system >s destrnbie can be answered To lio othenvi*e is to 
put the cart before the hiuve 

■ Implying a national examination will not address the issue* of ngid and 
bureaucrairc Hhool gavemanwe and struiiurr, lou'-quahty textbotiKs. and inadequate Hhu»is 
of educatt<vi Imptovmg assessment needs ii> be considered as tme pan of mtegrared ^ssicmu 
change 

The proposal calls for natioo^t boards to set standard'* it could create a natuvial 
school board that, by sefting cumculum standards, *i}i lead to a centrah?cd. national 
education system. Because the conse^|uence^ of such aciitms cannot m>» be knot*n. but miis 
include underm intng dem^x-ratic contrail of education, we should m« rush into that pnvess 

• Staff development i% centra! to school refi^. but ts not adequately addressed m the 
LRDC/VCFF. proposal If teachers are to teach to performance based a•.^essmen!s, to ic^t h 
the "thinking cumculum," thry have to know how to do so This mviilses developmg the 
ability of our nation^ 2-1/2 million teachers tt» teach and assess tn new ways To be 



rttc» ti*p. HhtHit rcti»nn nv.isi Jn. ludc itv a^iuf p.4rtKJp.iTu»rt <»t tb»»>s' \»h»> uiti implcmcm ihr 
chjnpcs Wc vannoi ifTiptrsc nf«v asvcssmcnts tm te.uhrrs. ih.mpc m>lhtni! rJ^. ami vjv iXi 
n ' 

SitUtd hir ihr sy>t«fm lo hp fair iTunpmg assessment will mit b> itself reduce inevjuities 
All students must he assured a ijtr «ippi>nunil) h> team him u> ntvV a fhin|.jnr 

cumcutum thai uvc% pcTlitfii)jm;c-haied 9s%cssmeni^ Addttionall.v. U)e ^txi\ wf ■ mj!;jl 
ma^ifcrv ■ cwjK! efKiHir^ge u»njnp and tr;Kking Mudeni\ ai'ctwdjng !o uho ran hcs; tir in<ist 
q«)t;kl> renih the pvi] Thi\ dan^ • needs to serw^usly addiessed to uy t<^ en>ufc 
jitniituffv ami pnv esses, ineludmg in ihe realm of ak*cssnwnl, ihai axe intJus?\'e and feduwe 
traiikinp and tvhe" kinds of vwling 

Wr simpl) dti not knuu whether it is feasible to <:<>nstm.t a natumal exanim.uum 
i>Mem l"he >fchoie pnxess. ponuularly ihc cahhfatKun. vcmld pnuc u\ be UKy complex. 
Cxpcn^isc and un^^jcU^ to wtvk f w example, Fngland rcfentt) dr\>pped a nuxJeratKm 
j^xess Inw Us nalxmal c»a") pax. ess hc&ause it was tiiti cxpensivr MtideraTKW is the 
pmrtss by uhikh icat hers help shape i^tand^ds and Icam to grade papers, pr^xJuLls and 
perfnTTTianiT* unifjirmly s<> as u» prtxfiKe consistent and rrljaWc results Ntodrrairon is 
valuable and necessafv ai>d n?usl bt mduded m any performant'e based system, hui d<Hi)g n 
on a natuwul level o.-» tup of stale and hxal IrseU may be hk> mush as ^rll as onne^ essary 
fw cducaiJonaJ improvement 

When the vtmipleMtic^ and expense of Ihe pn»p«>va] bc<.t»me Lle,tr. the p(»rEf<il»m am! 
pmjctiN ctmld end up being Trdutrd to very hmttrd exams There even ^.ould be a rtrtu'Ti to 
muhiple -choice and shon ans*er cx^ms Such a rtrreat wtnild base destiuttivc surncoLu 
effects and undermiw all aspetii of cducar^mal impmvpmcnt 

■ The pix)p«>\al n m>i cCHxeivcd ot as one part of an overall educational jnfonnatKvn 
system Havmg assessment out<:»mie mformatfon on educatKui is not useful unless we als« 
have Hdet|uai€ informalnw on mpuis imtwrey. leaihing staff, building: quality, eu J. pr<Ke^ve^ 
and pri>gr3m> (cumculum. msiruirlicmal tnelhndv tcxthmks and materwli, cUss wie. role of 
traeking. gtrvrmame and school organi rational sirucmir, etc }. and sddittona! outcome data 
(emptoymeni and fun her cdutancm of graduates. dn>p<»ui rate*, eu » Tbis mf<mnalK»n sh<njld 
be o&tatntd wtihout hannmg cducauim unhkc what has happened w«h multiple choice 
tests SchtxWs and prttj^rams ^h(»uld he evaluated on 3 svmjprehcrisive range of irtdt4;atOfs of 
their qualttv as c<«nmuna!cs ihai iupptm learning foi all Mudems 

Rcc<%mmendarfon% 

There »s ntjone. sample method of putting a natriwial education nrfonn pnxTss mut 
motion m the nghi direction it is a process thai can and is happening at all levels the 
classroom, the sch<ioi. rhc dismci. the state, coosonia that include alt of these, and at tite 
national level U is not and will not be a sm^xsh and easy process Hut as good practice 
becomes available in rrphcate. as improved cumculum and as^es'snrnls become widrlv 
kmfwn. as m;f nation's desirr to miprpve educjfum for all continues to pr<»w, then we can 
expect lo see real progress 
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The fcdf ral govfmmem can pmcrcd in <mr of t*o wavit t( tisn imfx^^e a nation4l 
tha! runs ihf nsk of shon-circunmg the prxcm of schwl ref<«ni Or ti t an fiiwl uays jt» 
suppon sthwf reform activities wttfioui imposing s nanonal test 

FairTcjif <x>nc)adc?i that ihc House of Rrprrvcmaiive* ih(»uld rtoi prnjx>« a naTamaJ 
eK&m cither mime4i)ate}y or to be m plxt wilhtn any fiKed tuiietiibie, &ui:h »% five ot ten 
»-:ar\ Rather, f^ajTTrsi urges the federal gpvemmrm to take the following swps to impnue 
education and asiessmenf 

• ■ M^isi states am. di&tncfs. scimg tn consoma. to develop and impiement 
pcflonnafKT- halted tneihods of assessment 

Assisi state and dismci teacher educant^i and staff development proprams, 
A^M%i the subject ajta groups. KtKh as those tn math. Eiighvh, sfudiex and 
^tence. to devetc^ and disseminate mcHicI wumcula. sijindands and asscs!kmrntik 

- Re examine the instanLes in which the fcdrraJ gmemment requires standard! ted 
tnultjple cK4iu.'c tesimj:. partieulaiiy fw the Chapter I program The iCMing rcijuirrmems 
\inaaHy Uwte program\ into being tesl-coachmg programs, though that, as explained ahtne. 
is a poi^ edocaiuinid melh<)d 

• Ixave the Kational Assessment of Educational Pn>gTes<» q\ a national intJicaif>r/° 
lo turn It mio some kind of a national test will end up destniying its curreni usefulness ami 
may pnxlutT dll the drawbacks discussed above In particular. NAFP should nm be used 
beto* the level of slate level CfHtipansons fair Test doubii» that state- lev el C0n>pans«»ns *iU 
be of rral use to educates and urges that state comparisons not be apprtn-cd beyoml tnal 
measures unless esipcnence and research dcrt^wstraie how ihe tompari«>ns will be used to 
impHn-e edut- at mn NAKP shnuUl. hcmever, include far more performance based assess menu 
and provide technical assKtance k> districts, slates and c<ms<)rtia **h<^ are implementing 
pcrttHTnjnte based assessment 

- Ctmsider how assessment informatttm can he « be mctuded one elemenl <if m h^»nl 
reftwi aittsiues an ' •*ne pan of an indu itor system, and not siesv assesvment m is<»lrfUon 

Only after . v .j educational reform processes have been implemented and evaluaied 
t'\ef a peruxJ of time shttuld the federal gos emment comwder * he (her ti is desirable or 
feasible to Imk the ne*ly developed Icval and state performance based assessments to eath 
other and to natumal siandards or cumcular ffaine^^orks 

Ixt us be clear F-atrTcO is are not arguing against Uiiounuhitt^y o/ for slowing dovin 
s<hcx>l If f<>rm Nor i< the issue one of the need standards Rather, the cenfmf issue u 
hovk *e defme educatKm We anr saving that we need nhtxil reform. n<»i more tesimg N\'e 
n-ed genuine accnunuhlify, nor re\r s^otes fnm muHiple ChoKe or short answer exams, and 
d<»n\ need to jump abciard an examination tratn heading into trackless rerram 

Our nation musJ not be misled hiI{» fhmking more tesimg ^t\\ solve our ediKatt^tnal 
problems Instead, vte must Construe r plans for refiirm that imlui?e assessments whtch \*;*n be 
osrd to help student learning, gmde eduiaii«m,i{ m>pfitsenienf, pritsidc mtt»rmatt<m ivr 
^liountjhihiv . dnd assist the g*ul i>f et^mfy. hut n»n Wink piugress at harm siudenx Our 
nation v^iU be f^ tx'tter served t" take the time to tlo the |uh %srll, rh in u» att havtiK and 
pi^orh «.tth dcvtructise results 
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Chairman Kildke. Thank you very much. 
Our next witness is Dr. Kelley. 

Dr. Kelley. Mr. Chairman and memters of the subcommittee, I 
thank you for having me. I am going to approach them from, ho{^ 
fully, two different perspectives other than a national one. I am on 
a school board right now, and its past president. I have b^n in, I 
think it is fair to say, hundreds of school districts in six countries 
on three continents, and hundreds of classrooms as well. I can give 
you my perspective better by telling you what I am going to do the 
next 10 days. 

When I leave here I am going to Schenectady, New York, and 
work with their board and their administrators to reorganise their 
district in two senses: they have to redistrict, and they are consoli- 
dating a high school. With a little luck and help, we will be able to 
induce and encourage teachers in that high schwl to take account- 
ability for that product. TTiat is very unusual in a high school in 
the iJnited States. 

After that I will have to go England. It turns out that the Eng- 
lish have decided this spring to test all T-yearolds. I am not sure 
what they are going to test them on, but politicians being much 
more practical than, perhai^, people involved in testing, all of a 
sudden raised a question. The minister asked, "All right. We are 
going to test these people, then what?" And nobody could answer. 
Now, the only thing I can offer them is that there is no necessary 
answer. But that is the next task. 

The task beyond that is to go to Budapest and try to talk to local 
governments about how to encourage entrepreneurship, and the 
reason for doing that is without a sound economic base you do not 
have the surplus in your economy to fully educate your workforce. 
By the same token, you never get that surplus unless you do fully 
educate your workforce. It is a vicious cycle which we would be not 
serving ourselves well if we broke. 

The first thing I want«l to do was to give you some information 
about how we compare. I say that because when I read about the 
national position on testing, which is what one finds in newspaper 
editorials, it is always about how badly we do. I think to compare 
national systems with each other is nonsense on the whole because 
TOch one is emphasizing different parts of its product, a'^d that is 
particularly true in a place like the United States and to a leswr 
extent Germany and a couple of others where there is significant 
variation even in the types of workforce being educated in some 
parts of the countries, 

I did include in the b^inning of my testimony, which I won*t 
repeat here, four or five tables and charts which will show you that 
we don't do that badly. I will say this: If you wish to control for the 
number of students tested, and on some tests, for example, the pool 
from which samples are drawn in West Germany is only 9 percent 
of their student population from gymnasiums. We sample from all 
75 percent of our IT-year-oIds currently in school. Whether you 
a>ntrol for that or not and whether you chw^ your test correctlv, 
yon can make almost any point you want about how well we do 
versus somebody else. For once I wanted to make a good point, so 
that information is there. 
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It is not that I believe that we do awfully well in every asi)ect of 
our educational systems in the various States, but I will come to 
the particular proDlems in a minute. 

The second misconception I wanted to clarify was the extent to 
which national testing goes on overseas. We are often told we need 
to do it like some other country dora. Very frequently. Great Brit- 
ain is an example given. I would be embarrassed, given the extent 
to which the British educate their kids at all levels of education, to 
make that comparison. If it mak^ any sense at all to compare out- 
comes of educational systems, there are only one, maybe two coun- 
tries that are undeniably at the bottom of the league, and Great 
Britain is the one that is undeniably at the bottom. That is even 
true, incidentally, with respect to very well-trained individuals but 
for an odd set of reasons. 

Countries overseas don't go out and nationally test everybody. 
When the Brite t^t 7-year-olds, they are doing that kind of thing 
really for the first time. They have things like O levels and A 
levels and GCEs and stuff like that, and we tend to compare our 
high school graduates with people who do very well on those 
exams. But people forget that on A levels you can get not just an A 
or B, but also a C or a D, and many people do. Indeed, most do. 

The same is true with the three versions of the French Baccalau- 
reate. Roughly the same portion of people take the French Bacca- 
laureate as get a New York State R^ent's diploma among our 
high school graduates, or among our high sch(X)l age students alto- 
gether, even including the droix>uts. 

So basically, when you start looking across countries, and the 
size of their countries versus some of our States, it turns out that 
very often they are t^ing sometimes in about the same way we 
test, about the same proportion of students on outcomes-based 
measures. And it is fair to say that some of these exams I have just 
described in other countries are outcomes-based measures. 

If you look at the Japanese there is even a more interesting com- 

Earison. We hear that they have an end-of-high-school exam. We 
ave to be correct about that. That is a beginning of college exam. 
Every high school does not take that exam. Only 40 percent of high 
school students are going to go to the type of college that requires 
that exam. Two hundred and fifty percent of Japanese students 
take this exam that you hear alwut them committmg suicide over. 

As you go into high school, everybody doe& take an exam, but I 
won't describe the system. I describe it in the written testimony. 
But it is not one exam. There are prefecture exams for prefecture 
high schools. There are city exams for city high schools. Some high 
schools have no exams. 

Further, when you look at the Japanese college entrance exam it 
makes testing in the United States look very good. There are 
people who score very high, for example, on the English exam who 
know points of grammar that nobody in this room knows who 
cannot speak English. It is, basically, these are the building blocks 
in their most simple-minded sense. 

I am very encouraged, incidentally, coming from New York State 
because New York State has done a lot of work on serious out- 
comes-based testing. This leads me to look at what kinds of nation- 
al testing we do in thr* United States. We test more i^ple at every 



240 

grade levei than any other country with three versions of the same 
test. The three versions are called the Iowa, California and Stan- 
ford Achievement Tests. 

One of the reasons we do this has alreadv been alluded to. In 
order to receive our what is now Chapter 1 money and benefits 
from various other Federal prc^rams, and to decide when students 
are no longer eligible for those services, we have to take these 
normed reference t^ts. You folks make tha^ rulra. we don't. 

Our particular district has found sneaky ways, using the New 
York State tests, outcomes-based tests which can be nationally 
normed, incidentally. These things are not at odds with each other. 
It is just that you don't want to pay much attention to that norm- 
ing. But since you require it and New York State has taken the 
enort to norm a few of its outcom^based tMts, we use that to get 
our Chapter 1 reading money, and then we have to go back to the 
New York State money, the equivalent of Chapter 1 money, and 
put that money into our math programs. 

Now, there is a lot of criticism of achievement tests and I share 
them. I think that a mcgor underl3ring theme all through what is 
being said is that we are banning to learn how to test outcomes 
pretty well and achievement tests or that formatting doesn't do it. 
But the criticisms are there, and there are two things you need to 
understand about th<^ tests. 

One is the Lake Woeb^n effect. If it tells you anything at all, it 
tells you that the average, say, seventh grader knows more things 
that are put on multiplenchoice tests than the average seventh 
grader 12 years ago, 7 years ago, 5 years ago. If you are bothered 
by the Lake Woebegon effect, iust renorm the test to this year's 
population. 

Secondly, what people never realize about achievement tests— 
and this is in response to a question asked earlier—is that there is 
very rich material there to help teachers. And again, the only use I 
generally have for teste is for diagnosis. It tells me what a student 
knov n terms of what they are supposed to know and why they 
don't know it if they don't. Every wrong answer is on an achieve- 
ment test question for a reason and patterns of reasons accumu- 
late, ani it is very important for a teacher to know whether or not 
a person gete a particular score because they didn't finish reading 
the section or don't understand the use of semicolons. That infor- 
mation is out there right now and we don't get it, and, obviously, 
then we can't use it. 

Now 1 am not in favor of going out and producing another test. 
The experience in other countries, and I think particularly of Eng- 
land and of Northern Ireland, is that prcKiucing another test 
doesn't change anything in terms of distribution of educational out- 
comes. On even vpry bad tests you know who is coming out at the 
bottom now, and if you have new teste three years from now, it 
will be the same people from the same places with the same demo- 
graphics behind them. This is known. 

Also, there is not a great deal of disagreement among roughly a 
million classroom teachers about what people should know. I don*t 
find that when I go around. I even find general agreement across 
countrift* about what kids should know roughly on average at an 
age level. You know, if they are being well educated. 
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To act in a vacuum, as if a million teachers have never thought 
about this^ is really duplicating an efTort that is already going on 
on a massive scale. States have done it; and, ^ain. New York has 
thought about this, with teachers, with testers, and there are even 
some tests out there. And New York is not the only State. It is not 
a mystery. 

Now I Btn concerned about having more tests for another reason, 
which I alluded to in describing my trip to England. If there is a 
test that will be used, and particularly if there is another test, you 
can't necessarily, either as the producer of the test or as someTOdy 
advocating educating all of our students to their full potential, de- 
termine what use that test is going to \^ put to. It is in the hands 
of the user, and there are many hundreds and thousands of users 
in the United States. 

Now, I do have some suggestions about what can be done* It 
seems to me that talking about toting, as contrasted to making 
our assessments, which are many and getting better, even more ac- 
curate, is putting the cart before the horse. 

The things I would do, and these will involve both items I think 
that we have to keep in mind and some specific recommendations. 

One would to support programs for children, families, agen- 
cies, preschool and elementary schools that will help all children, 
say, at age 8, age 6, age 3— pick your age—all of them be able and 
anxious to learn, and schools be able to teach children of diverse 
cultural backgrounds effectively. It is not just the child's responsi- 
bility to be able to learn. The schools must be ready to teach this 
very diveree set of children that we have in this country. 

You have to understand that no country has ever tried to edu- 
cate such a diverse set of students since the days of the Roman 
Empire, and, obviously, it is something that everybody in this room 
seems to value. 

The second thing to undei^tand about that is that when we do 
this, and we have to educate all our children, even children of ille- 
gal immigrants when their parents are in jail— that is since 1984— 
it is going to cost more, not the same money, to prcKluce the same 
outcome with that very heterogeneous group than would be, say, 
true in Germany. In Germany, except around Frankfurt, there are 
two things that are true of a German classroom; they are all white 
and they all speak German. Try that even in Ithaca, New York, 
We do ESL and transitional bilingual in 51 languages. 

If a class of 30 or 35, or even 40 or 45, in the Japane^ case 
works effectively for mc^t students, it is not true here, if you want 
those diverse students in the same classroom. The best teachers 1 
have ever met can handle the low twenties with that kind of class- 
room. All of the studies of class size have been done after you have 
imposed this very funny tracking system that has implicitly been 
part of American education historically onto the schools, and you 
are dealing with reasonably homogeneous classes. So many schools 
are self-consciously moving away from that. You cannot use the old 
studies to justify large class sizes. 

You also need to understand that teachers work about the same 
hours as other professionals. The teacher is the only professional I 
know whose real wages did not go up between 1972 and 1988. 
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I, obviously, have in here recommend continued support for 
Chapter 1 and Head Start. 

Finally, provide Federal support for technical apprenticmg after 
high school Only in the United States and a few other countries do 
we keep all of our kids in the same school. These tests that are 
riven to some kids earlier, and they are not always national tests— 
m Germany they are state4>ased, sometimes grades in school and 
fourth grade matter in some states in Germany as to whether you 
go to gjrmnasium or not. But what they all wind up doin^, other 
than a very few countri^ particularly the United States, is phys* 
ically putting the kids in different places at an age where the kid 
can't make that choice intelligently for themselves or understand 
their consequences. 

It has less effect there, because the ratio of blue collar salarira to 
white collar salaries is higher in most other countries than in the 
United States. Alternatively put, income inequality is lai^er in the 
United States than most other industrial societies and is the only 
one in which it has been increasing the last 10 years. So, if you did 
that with our children, you are making a decision of immense con- 
sequence just in terms of how— what kind of food they can buy. 

We don't do that, though. We keep them all in the high school, 
even though we covertly move them out from the different little 
groups. But we tiy. But, obviously, they don't all get through, and 
the problem we have is, in my eyes, different. The problem we 
have is that we do have 30 percent of our students in the system 
who are not prepared for the workplace or further education. It is 
not that everybody is failing. It is not that we have a horrible 
system. It is not that most people aren*t making conscientious ef- 
forts to teach all students effectively. What do we do about the 30 
percent? And basically, I have been telling you, if you think it 
doesn't cost money, I think you are very mistaken. 

I also think you have to own up to what the problem is and not 
think that a national test or people sitting here, particularly people 
like me, can do anything about it until we get out into Ithacas, 
Schenectadys, San Diegos, wherever. 

There are a couple of particular things that would help, though. 
The German example is very good in this sense. What the Germans 
do with respect to thc»e kids they do separate out is they work 
through partnershii^ with businesses, as you know, but in the 
process of technical education, which I would advocate beyond high 
school as an entitlement, they continue academic education. Part 
of their apprenticeship is not only in classroom work in these 
skills, you could rail it essential work skills, but continual educa- 
tion in reading, writing, math, et cetera. There is no reason we 
have to do that for our 16-year-olds ^eoing into high school voc-ed in 
New York. It ought to be true for 18, 19, 20, 21-year-oIds as well. 

Now, how do you get them to stay there? Actually SCANS can be 
a very big help on tJiat. I have seen a number of the SCANS work- 
place scenarios that are very well done. What I plan to do with 
that in every school district I get into is take those workplace see- 
narios in to business people and the school board members and say, 
"Look. The first thing that has to happen is let's take middle 
school kids because that is when aspirations and expectations di- 
verge, and let's show them that what they are learning in sixth. 
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seventh and eighth grade is exactly what they have to know if they 
want a job." And we can do that by using the SCANS workplace 
profiles. For that purpo^, they are extremely well done. 

The employers then around you are alerted that in fact your 
school is teaching those skills, and the kids know that tho^ skills 
have to be mastered to get any kind of job other than flipping ham- 
burgers. 

One last point on this. I would like our business people, and we 
are working on this in Ithaca, to use high school grad^. The big- 
gest resistance I have found there and elsewhere is a general view 
they have of what I would call the maturity of the high school 
graduate, not their competency. You will take the ^me kid age 18- 
19 with the same transcript, an employer won't hire them. They 
will flip hamburgers for 10 years— same transcript, no intervening 
training— they will hire them. 

There is an age bias out there too with respect to young people 
being employed in our workforce. Employei^ don't believe they are 
skillful. Involving the employers in individual school districts with 
SCANS-type programs will educate employers to the fact that they 
do have skills. If a national test comes along in that area, I will do 
my b^t to make sure it never is given in Itliaca. 

Thank you. 

[The prepared statement of Dr. E. W. Kelley follows:] 



ERIC 



244 



est: Nativ'TiO; IfrsXi* AjffCt t hr Quality c-f tuu^cj* 
Test .m.>ny "-^er /r*^ criirni V r. t: i rn.-^?:'^ ^rv 

anc? La\>jr Marc ^l 14. 

Corrx*! I Universi :y 



I;...,.: v.;::::;.; 215 



.,-^;-t;iaj;inl. ti-ibYs;. -^c^v tc-n-.e in --civ^hiy thj4-«f ■fc>rni©: tiistftv thfit 

. ^:^v 1 . IC? iti^xr iti^t,*.^ l^U«aX vv«ff*p*t6*AC*4€: trri i*?^.•Jkt^J 

.-.^Sk^fo^ij^S^. i^y t^fc.tr» txf!^».c»r count r^y Cift sojn^ taEic ^ficu^t^L to h.'^: 

■■ iv..j^*.-pi^tti*l lyttx sov*v^* fc'duc&t^C'rt t^^' United '^.t^^tas ir- 

V J «* t- .^-'r. c****w '"Us '.'r-: itju l«taft»r^- id t''«fc >n*'j>':^*& •_• ' 'rt07.\ . 

^w'' ^-u r c w-'-'if^- .W4c*i;lv cC'IhS^' .'ror.. ^--fnc- f-c^v j i : c^^- 
C^^Irr ^ar^T-r^TT ^: :- e. , "•riirf.ir.^ ^ly * ^ 



ERIC 



*m COPY AVAIiAaLE 



24 



246 















0«4c 




riM*nriMi 


















POP 


POP 


»^ 


POP 


POP 




«JP 






( 


» 


t 


« 


f 


K 


t 




Toe*' <30»ym— i««eh tMt 




















14 






13 




8 


7 




Au«trM ... 


3 


2 


MA} 






WAJ 


CMAI 


IM 




15 






4 


fi 


& 


WA) 


fN 




11 


« 


» 


« 


B 




QiM 


IN 




fj 


4 


3 


7 


7 




CMA» 


(M. 




B 


» 


6 


8 


3 


« 


4 






to 


to 


(MA) 






<MAI 


WAI 


(M. 




7 


tt 


««A» 


(NA> 


PM> 


CMAI 


9 






7 


6 


tt 


10 


<MA) 


CMA) 


MA] 


(M« 




(MAJ 


OCA) 




f»«A» 


(MAI 


MAf 








CMA} 


MM 




? 


(MA} 


(MA) 


MA) 


OO 


toNf 


f2 




to 


17 


7 


• 


MAI 


CM/ 




4 


«MJ 


t4 


• 


(MAJ 


f«AI 


f 




MmMMtf 


1 


f 


1 


tt 


t 


S 


t 










WA» 






•lA) 


MM 


(M> 






e 


? 


3 


(MAJ 




MAf 


<Mi 




8 


a 


7 


ft 


4 


4 


a 






t« 


7 


%i 


1 


• 


1 


• 























MOM DA WoM. . » c h iiiw wFir *• ii fcA ■ Ct^m ^— > T««#nra C ifc m . C ai m ii l i 4**-, Maw V«*. Undwv 

»8W 



The l .f.. {»af, ihr l(»wst itttiitioo rat«t* tfHh Swpden and 
inpart < U'H*' t«> thi^ t .S. Th* othvr d*?v4*l«>ped countries havt? wuth 
higher attrition fjteh. ThuM, lh€^ verc pr>or nhowlng of the V.S. 
u;> f*ij? I HiMtf riinVlTiRii lb al least partjv due to different inl 
attrition. The data underlying the ranking fpr POP 11 tnrorporate 
<»n «4<l?ust»ent for iht- «rffert« of dif f^-vpnt Idl ftttrJtloo on te»t 
fccorcs. However • the adjutitiDent pro* Pdwr** is biased In favor of 
(i?MPtrlv» with I»w at trie Ion -I.e., the t'.S, -and tltert-fore 
.>»-rr Ad}4.».ts for ?he bias, "Hms thr true reUttve ranklnR oi tUr 
{ ,S. is h»Ri*wftert to b«-twe*'n the reiiultB for POP I and POP 11. 



247 



Enndlnmt RtlM for Stodcnts Aged tS-tS and 2&-24 







AgelS-SS 










Omiry 


Iter 








EmoUrtf 


§Umkmit 




* rwr 




44 


IT/^ 












7 


1971 




Q 




197D 


78,1 — ^ 


2 


1970 


285 


2 




1990 


51.7 


9 


1972 


237 


3 


Rnknd 


19K7 


47.4 


10 


1972 


13.8 


13 


Ranee 


I97D 


543 


6 


1972 


170 


10 


CcfTMsiy (FDR) 


1969 


30.* 


16 


1972 


17 1 


9 


bcUnd 


1971 


47.0 


11 


197D 


122 


14 




1966 


30.8 


IS 


1972 


20.1 


7 




1970 


37,4 


U 


1972 


19 


16 


NethefUnds 


\m 




8 


1971 


21,2 


S 




1970 


ee,9 


3 


1972 


19.0 


8 


Sweden 


1972 


68.1 


4 


1972 


22.4 


4 




1970 


62.6 


5 


1972 


103 


IS 




1970 


39.4 


12 


1972 


15i) 


11 


UKslBO 9C«et 


1970 




1 


1972 


513 


1 



Sfifiir SUMS lor 15- to tS-retrcId* adtplcd Cm UhfC^^ 



llir IJhiled StetM ««raQs » higher pmeftfi^ o( 
dun d6c9^ ccfaer muttm Atfion. 



Frotr: 

Michael Kirst 

Who Control s our School^ 




248 



Ad lottnuAkmil CecnpiarifOfi of Dnt Scorn in Science 
of Students in Urt \tu €d SccondMy Sduial 



Meui Sdencr Scorv 
0 to 7D SD 40 SO 




Smr. L.C. Comln ^ ^ F. Km«««, frintf £tf iiotfiM w 

■ OwKnDScm ■'Rif>9% 1^5% OlEiplIb 

imclvis^ tHf Utited States tie flkm«d by «gjbt*njnism of the 
U.S. •)rtmn. When fcoir* eompufcd m MtfuUed to icAect companble ^)to p o» li ons of the 
fdmnt tgp gfoup^ U S^ safics are flotfiu to those of othfr oxuttnM. 



From: 

Michael Kirst 

Who Controls our Schools 



ERIC 



249 



An intmutioiial Ccanparisoo of IM Scorn in Kfathemaiks 
ol Stiiiteirts bi ^Snr of Sccondaiy School 



10 



Mnn Mftthematks Soorr 
15 20 » 30 




Cotoffm (Nrw ybtk: 1967). 

■ Awrs^ MAkhenuiks Tesi Soote for Equat Pioporiion of Age Cnpups 

■ Avmge MAthenvitics lest Sccsv for Ccwntiy 

When KOiVS of #D students studying m<theiiutxs in the tmninat ol scvondAiy sdioot 3rp 
compmdL the Utitw! States ranks t*$t, m port because U S. st^toch enrofi « mudi htgher 
peiwi4>gr of the relevant age grou(> in mathenutcs <I8 |;>eaent « ^mst 5 percent in some 
&iropMn axintries). When scms oi equa! pfoportkins of the »gr group are cvmpared. U S 
students perfem abovt as we^ as stucknts m Austnba, Bdgium, and W»t Germany 



From: 

Michael Kirst 

Who Controls our Schools 



• THE 'i^£$SON^? HRW ABROAD i 

; Ji;«pan f»fe^ «tet;vf>«*i ^xafifS for ifffcit f^'^v hif^ ♦ K^^o 

^choolfe c«^^=-rAt49<i t>y. fc'if f «»^t pufelxi: iiAcior author ili*tj 
^ifci^fcpt fiti-wi*ni« :3a6©c larfieJv*on tHi& r©9.ultfc of nigh cchDol 
.-^niVio^ce fJ^st schcolfe &hr^rg«? a fc^&t ^^ul *rf, print iP'^ X 



¥ypw C'f LC^oc»; Percsnt to Vnivtfreit ,^ Percerit to Coil^^fifr 

-City ^K€fOe> fecedemxc 

City cciimierc I a1 ^ ^'^^ 

City night tecriftical 1' ^ 



Ho%i of the btw<*3frr<tB who wi:: get 
pr&f^ctwr«il ^c&defiiic high school m 

schocis* 



into Tokyo Uni vera! ty fio to the 
A few go to city fc*c€<^e«iiC 
to r&ther ri&»*^, ©lite, c^^ivate 

Japan's Higr- Schools 



BEST COPY AVAtU^LE 



251 



aliJiC'St neve-r ^>:j'-':»f-.^"i t-:-. t ut y^t ix t U' Cw-nv^rs*^ in 

English at ev*-rj a pr^r.-iltive >vel Many .K^panest- are faiiy aware 
tJ'iat tneir Uh*i.n'^\ e>:atr. .^yrt^f;* ri -t cj;w.3jyr> iff^deurt- *i*v^n t^:^ 

skills, let alone morf* complex pro^ierr. ssoiving abUiti^s that are 
lifj?' V3;Mgr;'=- T. t^" v.i-^-.A ^JVi I^^-'T^N*. - i r-Jiii.'! Vrr V♦^* •tjr:'-? 

is a rtrong press. ir^- vfor r^asLT^t net glv'^!. i^^rr^ 5 
f::ucifrri3 to ^it*.^!:. ^ r.Igh >vfi of tisil. sX r<»:e anJ r:'i:«r l.ui^ :^-y. 
academic skills As can be seen in :i;e av v^^'inj. any i ri^^ dijgrs3;u .u\y 
a niir.orlty vr .'ap2Tiosr students age abc^^t :i fjKe their 
"nationaJ** exarn used to screen into ujii ver^j i ! ies No Japarjf-be 
exam Is Otrslgvie'i to asses? their educailv-ri of a: i students or 
give the country a rc-purt card T}*e saioe is true if: Fra:,;^ vu*=-r- 
thrf t'3':':d>'3..2r^^te li? i?; i 11 tai-.en by fewer than tO% of siuc^entJi it; 
their :ate tee:.£;. Actually xh*^ bac c al ai.4re-^te 1^ ^ uuiu^rt I'f 
different, overlapping ^.treanis of course'i with curriculun* fixev 
hours of instruction and final exams mandated nationally drades 
are glveii t-: f*uaenti: ae> a CunbvqueriC e of the ex^m 1:, eart; 
subje'Lt that forms part cf the particular baccalaureate tli^y 
taih'.e 

British A .•^vei ^-xai^s are p'^gge'.i a\ iro^nd tfje san^r lev<-: 
the bacral a-iredte, ouX allow greater students ctiC-Jce in c^^rjiJ:! ? r^f; 
Svibiecte Wt,i,*- tJ-.ere are thr«e /h^i-jt variatio?;f ^'f t^.- 
baccalaureate. British students i an mix the A -.evei examb tney 
tak.e In Mia?jy ways Ah»out half the per*:^nt^tge ^f s'udentp in 
Britain ta>.e A- levels when com^^'ir'^d to Pren«-h students ta>:ln^: 
some baccalaureate This is consistent with an almost universai 
observation: if comparing the products cf whole educational 
systemt' made iit^y b^nse at all. Britain \t probably at the bottcrri 
of the league of indust r 1 al i::ed dentuc rac it-s. Most students who 
finish secondary 9,cho:': f^ever taKe frK:»re than a few 0- ]*'vel exan;^: ; 
these exams can be passed by mort li-inintally ohip^: 'r:/. V T> fil^f. 
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school gr actuates 

The iev*:*. at ^••iiUS th^- ifr;*:st. arvi Fr t^nrh exams ar^- pltrti^o is 
about that thf» average eno ^.'f fir.*^: year exaft; at q U S 
a:a.:eiiil>: ir-^r y'*ur : . ,1 *'^herr it ^C'ri.-e vtirlatlon dcro?b tfieee 

Britlbf. etM.ient »U':.<r^e4u. t!;*:' A i^vf*. math wouiu ji-r-iUabiy 
kfjow more fyjaDi-^fma: Ic s than t}»e U.?.. four year college frps.^*m»^'n. 
t:'*at buV&rX oi free.r.iMei. wh* v^^:,! inavr ir: mat?* ano s:len«*e. the 
fair comparibon group, ' 1 Know atx>ut as much However, tt^ere 
i^vulo he twUe the ^v^'por^^'*^.:) Pr-n'^h a:Ki U S. student.^ 
involved in the i omparison 

Like Bri'ain aiid Franre. G^rrmany has no national assessment 
or any n^itional test that a:ntost ail stuoen^s take^ Like tfte 
Uidtea States, Cuntrol jver educationai instruction aoU 
evaluation lies principally at the state (iaender) and lo^al 
level As in the United States, frequent professional contact 
across states and a national media result in great commonality of 
curricula arv^ ntaterials exartiined by state or local educatic^nai 
authorit ies 

German f'ates vary in the age [11-143 at w!iich tt^ey first 
fc^rmally s^^parate students into more or less academic trai'^s; no 
unifc»rm ^iss^^ssinent prc^ess ir; used across the country Fourth 
year ^jracies. sctme exams 'and teacher judgement (rarely parental 
Uidgement . uTtMke the United States) are all possible elements of 
a decision to track a child to gymnasium Historically the 
possibility of movement into white collar professions is largely 
determined at this point, the child can hardly be said to be an 
active and inforn)ed participant in choosing his or her future 
Yet the sifting process is largely not resisted for two reai^ons 
Study after study snows that students around the ages of 12-14 
absorb the expectations f ratner than aspirations) of those aroutid 
them, including, critical iy, those of their peers. These 
expectations are largely based on the historical and inexpensive 
inforn*ation provided by parental education, lob status and income 
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kfor the U S adci race an<i language variation) Seiond the 
status. aiK* fmanvlal penalties of not o^ing in a white collar 
i..rot ession is nnt nearly as great in Germany as it is in the U S 
Tne Germans hav*=* an excellent apprent ict-ship system run by 
industry and stattr governments together Fart of the program 19 
i^vhoo* I'df^evi Prvducts are largely literate workers with ski ^ is 
to work productively in a particular industry Theee workers are 
weii j:aid lalnio^'. alwayr a:j n«jch or incre Xhtm their U S. 
counterparts evc-n c^n tlie cccasions w*ien U S workers are a? cr 
iiK-re product ive- Inequality of earned incorm* across whiite 
collar* tiue collar workers* managers, etc . i,^ lower in Gerniany 
t^lan t!ie United States; only in the United Statv^s have ail 
measures of income in:*quality increased in the l.^st ten years. 
Of course, in Japan* inequality of earned income is the lowest of 
the countries we are considering ar^o Xhe percent of the economy 
totally in the private sector is highest. 

Nune of these countries conduct internal assessments like our 
N'AEF A.i are involved In multinational survey coropariscns. niany 
<j't w^jlth involve the United States as well The Council :»f 
Eiurope Strasburg probably has the t>est information atx»ut 
national and even local educational practices and tested 
i ^v: K't in^u* ts N\ ne of t^^*=•se cc-uf.trlrh; has a single test that 
a.: siucJents ta>,e The Japanese efKi- c»f -high school exam is not 
Ti5Ketf t*:- gi'Mod'e- rattjer it if t^^K^n to get into college. 
Aimc»st ail of the hig)t school gradu^t^s in France take some ^orru 
of the baccalaureate. However tfjst number represents not much 
higher a proportion of students tnat those in New York State wki«'» 
receive regents diplomas 

fiATIONAL AND STATE TESTING IN THE UNITED STATES 

Si'A.ozls in the United States already give almost national 
tests T^iese are the Iowa. Stanford. California achievement 
tests I'Chool dihtrlits must us^ these or other natiojially 
normed tests to determine eligibility for various federal 



256 



i'-r :i:v-^:;r 7! i t'-^'*- ti? «. .ti j -rir 4( ^ . 3TXi ar** i^ts 

-i-',: rati . «->:ji. n w-v-r M*-\ riut-i:' ^r^iti eval.Jwt^ e^Ci; .'i^l »*:*l"lt in 

• . : • : 'i ..T f* 1 ' .4 . •ah . r : r. r"^ ;' * 1 v ; "ir ^ , m-.» : '.H n t » 

f'urv.er n» *:* . "i-^t*- w** fiave mf^nt .c»n*:'c*. schiev^'ment 

testt are pairs ive. L-ubt-U* :^^»t;•f:* njuitipie guess, tinjed exani.s- 
T'l** .;uri*.l::. * *■ r . r . y .=1: ••.'-1:;' 1f>*:' rxTenT t-:* wtti^ti 

vrijjl v.- a :v y -f '.OP., t'\A iwisx await the 

p-'I;*ic<^l ^cjTr*ge t r-t- explicit about iconics and t^ie 
-f- - ; /i'.rjier* : ] ^r: >v:u\i:..'- ?:aced testing. The LaKe Wob<*ggon 
<rf:*^;t 01 a:?.>-v^;j.-*.r tt-i.ti x*r'.:b i^t t.Sat rrtu^.ieins on av^rag*f 
f',! ; , w iw*.' 1 ' : .'i' . L- : ..-.s- ■ 1' t S'^ 5. dfifr dge did five or seven r h 
ago Tjif Lahr w.-.t-egg; :i -^^f*:*:! c -^r Lx- r : imiriat ^'y simj-'iy 
r-!*. rn.ifig t! ' T?;':* a* .'»l-.-ve;{te:-t teii ts are given !nr«re 

studenti: ttian NALT couic- evt-r sample People are excluded f rvm 
*. :v. trits J . r ^ 2UiT- rea/^^c^ns ^ 1 t'orrie tyi-'e of handle apj.'eci 
cc ndl t iorii: ' NAEf' oversamplirig wouici be unnece3t}ary. if 

^.rvrti.-^nt ^. : r^f=; wei>'- c*t-ing comparfd Ac hieve'uerit tetJtb 

contain TLir.y w-r*^ .* .^^.^ 1 1 '«n^i ai>.; .^rf- drgiatly irKTe closely tieo to 
w.\.:a* r .'!tiv e-.:-.-. . r ^ V.Lr:k I r ! Iciren p.hoaici KnC'W after a nunV?er ^f 
j^-iit '.^f L'. ■ : *r:^; a^- .oj. - Ji Hrp-r.::^ wtth n.vrrj.iP4: a. h;ev*-ijt^»/. 
-^v* c., .r*-^ dr; ■ t!.-- t^.'r^; 'as ; :" they wf-re outcomes base^i on a 

j^eneral turrioiilfs Ocioly. we cJc- not uiie achievement tests to 
:.e.i: ted'ti >;n.r evrry wr-.^ng ari?:wt*r Is c.n a multiple gjess tect 

v3luat':e lor teaching 

h> -itt 1.. Aj.-i • ' Te:;:, n-. t r^t5tl^«nal testa that 

measure out «'t'mefi. even tJi,-^.^gri they arc inc iud^v: in the Depart mef-it 
^f L":iUv at **w^i ] r*i.irt** .'ATs csn only be considered as 
pr^-cJictors of oc-. !t-f tdte ac»3oent-c {.'^rf ormanc e. l*ut they are such 
tr.e: f 1^: lent ^ reoi. t . r:-. we nr'i'd nt-t con3i<^er them farther 
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IrKiividual U. S stales are often as large as ii*any of the 
countries irith wnich we compare our t^diicat ional outcomes- Our 
larger states probably achieve fuii economica of scale In 
evaluiatiitg student f'^tr formance State minimum competency test? 
cto not demand sufficiently high levels of mastery in any subject- 
One reason the U S. has more high school graduates than other 
countries Is that the comtrion denouiinator of graduation la a low 
level of acct'iM;.! liilai>rnt at academlr subjects Yet some tests, 
like many New York State regents exams require ;eve]s of 
pei f cTinanc e of students that would place them toward the top of 
their age peers in most ot^ler countries. New York has also 
developed outcomes based tests which compare favorably with 
diagnostic performance tests given anywhere (Unfortunately these 
are often still in a multiple guess, bubble sheet format) (See 
tablea 1-2; figure 1) The problem is not in the existence of 
ever better and w.^re diagnostic tests, but in the fact that a 
numt'er ot stuoents do not do well enough 

We <.an conclude that the United States tests as much as any 
country (For some guidelines about testing see Figure 2). 
FToba^Oy some state tests are mc-re sei f -consciously based on 
educationally desirable outcomes. Our nationally normed 
achieveuient tei/ts are taken by a higher percent of our students 
each year t)jan ta>;e any tests In any other country- No other 
country has NAEP on such a scale for so long- Two caveats^ 
almc»£t everything we do with NAEP can be dc»ne more accurately 
with currently given achievement tests: wf»y give the NAEP if 
there is no consequent policy change at national or state levels 
that will effect educational outcomes at the classroom level 
InfL'rmation is a costly waste if It is not In a form that can be 
used af>d then is used to change outcomes More national testing 
in the U S will have a feww ecrtain results- less instructional 
time and more professional testers employed. 



mSlI^IFirXNG THE l>ROBL£IS 

• . "NMionai tests are not the protl^sm witn U S. €ciucation nor 
are they the eoiuiion to any pMbl^. Our prot>leia is not that 
education -Id overwhelmingly fa4Una ua- Rather aome of iia are 
ml beine \ ^y^^^ .^^ Bct^ol ax^ m other i^roviaioQ ia tnacle ^ 

heip thwe people be useful in the iKjrkforce cunllKe <^n»any 
.^jand. Japan)* Soto of these people even receive high school 
.:^pioaaa. In the future our wrH force will require a 
distribution of skiUs we are currently not providing in our 
educational aystem- (Figure 3« 4>« 

^ I6)en. students are tr^iked out of academic education in 
6erttany« they are aijspiy dropped^ In Germany all students have 
access and the later differential between their if^ome and that 
^f white collar workers is smallCsimilarly in Japan)- Students 
;pay little or nothing for this training* cost is shared by 
federal and Caiostly) state governuient and business in Germany. 
The savings occur in later tax collections and lass on tiw Job 
training* The apprenticeships all involve continued general 
train! j\g in iiteracy, writing end numeracy. The two most popular 
apprenticeship programs in Germany ere automechanic and 
hairdres8ing» the same as In the United States. 

The distribution of those i^ ^ not achieve high levels of 
educated competency in the U S. is well known* Tfwy are 
diproportionately black, hispanic, and low income* <See for 
example Figue 5). Eighty seven percent of children under three 
born of black never married femles live under the poverty line* 
We already know many of these ^ildren will not attain high 
levels of coiqpetency in scrK>ol for a conq;>lex set of reasons none 
of which was initially a choice the child made. It ia unfair to 
blatte the child for events he/she did not choose and insufficient 
to do nothing to make that child a competent citizen with skills 
to enter the work force. More national testing will neither 
io^rove that or any child's chance to become successfully 
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educated !K»r wiii ttw i>attern of failure t>e changed' IrKteed 
changing adding 1o nationai testitig can oftm ej?9'"'?rbi»*'? 
ine<iimUty opportunity^ :v:: -a- ->'/ .. . 



' T^"* . '.ir^uu vv cvXiu-u*..*.nal ctitcciB^& In classroomft and schools. 
2.. Subpart tlwse programa for chiidrerw famUlea, agenciea « pre- 
/^hool find eieinentary schools that will help all children at* 
aay* cise S be able and anxious to learn and schools t>e able to 
teach ^iidren frooi diverse cultural backgrour^ effectively. Do 
^this without Isoposing a sionolithic culture on families. 
VtKterstand that we are the mast heterogeneous culture ever* with 
the possible exception of the Roman Ea^ire* 

3. — Understand that to educate all our children Cwhioh we must 
aince the Supreme Court formalized this practice in declaring 
Section 2i«03i of Title I of the Texas Educational Code 
tinconstitutional) will cost mare than would be true in 
.homogenous cultures. Class sizes of 30- 3& are possible if all 

the students are white and apeak Germsns class sizes much above 
twenty are difficult for a skilled teacherr if the student 
population is heterogeneous. 

4. Understand that teachers work the same average hours as other 
professionals and work at tasks most of us could not do* 
Teaching is the only profession whose real wages did not go up 
between i972 and i988 CNon-professional real wages went down in 
the same period). 

5. Continue support for Chapter One and Head Start. 

6. Provide federal aupport for technical apprenticing after hi^ 
school » Only in *he United Sates wsi a few other countries do we 
keep all our children in one school through age i7-ie. Children 
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gro¥. chaiifie tft^ir 4nt*rMts and progressively MKe choices for 
■'i theaseives imuch j»>re t>» HGermana for eKM¥>le>-"^^ shoulci 

not"be makine choices for youth that they Bhoulci aake for 
, thamselves. It is they imo live i»ith the consequences^ 

Be sure that increased general literacy* writing skiils atKi 
■ '^numeracy are part of the program. Provide incentives far states 
1„ ..to organize the programs atxi insure that it work on site at 
^lj^^,^)a^ipi^^eB as well as formal training. 

-..HsKe-.it free or almost fr^e. Con:i:i-L v.- i \'\'e'.c^y*^fi<^. '..r 

; eHCei;frr.t i/.^^.i ; rvi^i ' s^cre r'S"*^"' i'.^. t^.-i^ *-.'=■;' i-^** 

-l*;"! -t^^a: «t> , " . " > ^* . - - .'"^ - . 

" • iv*. w.iUTtgc '.w^ia.* educational outcomes or their 

distributifrn in the population- 



;er|c best copy Mnium 



TESTIS $C^@XJl£ 







wnmNQ 


MATH 


SC8CE SOCMlSTU)l£S 




Ototatkan 




Cfieekltet 





FffSt 


Okrttlton 









6«oond 


ORP 




PEP 






rep 




EEP 




Fovrth 


ORP 




'pp 




FtHh 


DAP 


EEP 


P^ 




sum 


PEP 


only 


£££ 


PET 


Sov«mh 


OftP 




PEP 








PCT 


•TBO 




Nmm 






HCT 


fttT 
















fCT 




gCXUS'HIsiofy 



I » 



■ To bo detonnfn&ti^deveit^wd 



Par - ftr/z^i/siary if 




262 



34 DRP Uniti 

T>»y est bwg» TWy «ot bc^** Th*^ w^l hor*^ TKrr 
•of fi<h. too f»e<i o% ip*\ng They t*»<l 

UfnuTwf TS«r K(*d <n the foH Seon iocA for kM>d th»<^ 
T)^ hwfrf Th»r H»h The> »oon Thr> pK'k bet** 
T>»y cof o iof T>*»y gfO- 'oJ Soon, comrt »» 

cdU ^ tno^ &tff don t ^ dut 

Th#» aio« t J<w«^ food THtf ^<3* f /^o*Afh ^Hry co" 

3t 39 DRP Unit* 

i o'» t'^hf Bk/i f*»ry o'f v^'o'^g '*^eT ♦*«lf> «x>i» 

t b«*£jJ b«i' f<.rtl A bi/d ton ft>#ni «f» «rac»4> 

j di'K^^i Bwd» rnone rt^*^ we>^g\ ^^wQ'd o-^ cJo*^ 

43 DRP Units 

t otihp'&i ^o'-^c'v bvT •'^♦r'" ^Vir o'* »o'i *^o« 
I ^♦^c cr »t bsiffo C"^ mt,;^ *t»ttf d-**rfen» c*oi» 

47 DRP Unlti 

Tht po^ o b««< f» b« *>frf in stnd low »«dc « ccJted 
1^ matf^'e b*9d1 H « txyn* to ironr plonn Of«d 

n© pro'tftfO'* O90''^Jt mnotf* of th» OOCO^ing •'0»<f» 
So#n*o*vwieHii^(»*b> rf'^-«>§KoiM fftfV T>»ct 
i'Op »Kcf» ho*Tie» t/fvi»r yoi^ftd TS* iwvtf ftc*** w-** 
no* rHrw ou* te ««o T>ier to^e 

51 DRP Units 

iool*'? or rt* y Qvfkd D<sy»*^OT^dcicr^d*-tipo^ 
Q f«m« Ai tf« to^{ onto t»Ov«t, fhr-j» mh«v3 hfce 

c«Ovg^ tfo to rho* tSe woUiift^ »i*<ki *.M no* d.t o*/* 

S3 56 DRP Umfs 

Thf p^oo e o^ G rM* H>* oSpNs&r* **** '^'rv 

Kf hut O'tril mro^ ho^ '.ght to »<» 



nght m filt«mi3t»ng tmew 0*T*k naw» for iS\a 
%y*%9frt ^ wftfiftfl const *rc^ fhso wordt for and 
'htm ' T>m mvrfiQd mm^nded o* c*»o gofng bod 
tprth, pk»w«»g o Wld f>«*»HiO**y- O'^wi* •ffC** 
on}y m one d'«VC*TOn. oi m099 pecp^ do now 

^DRP Units 

lha ou'fO boord a o >jmpk recton^ubf wocd 

ocroa Q kytg Adga Ths tto dv^in ond the worth *fc«.* 
* fto," oftd 'goocSt>y«' o*p«ir b*t<» A jmaU 

mov^ipd ort 40>ter» «o *< ff*cy*t cov'y Of boo^d 

*5pd« orovTid M mc»^ oppof**^ •Tfthovrt o^f 

64 DRP Unitj 

aM*04pKcr< cHon^ A^tho^OiOipvitt know Ittai Htiv* 

f*>ip^ iry to d KO^^ b*'o** op#n*n9 0 to^nb w+v^ttx* i 
Hvf m*tfO>& Spcc«o' tooH hovf be«^ d«-f<«d 

fK.i pwrpot* On* ot tHe mo^t trtrJ^J ti o k*vi o* | 
tfome^o '♦vot CO" b< <Vopi>*d f«*0 9'Ow<Mi br^off 
d-^^.f^g i»o^ H t** cc»«r^ i^dicofrt tf»^ p'rir^vr 

wsH O*^ K>rn(-&^ <0<^ p'rpO'* tOtoke i^ffJX (op«#>«<vc 

73 DRP Units 

K»Wlf*»'Vt* iittfo*v*9 iJ»o*»d en tmMfi>* ^ *idwdwoJ 
fMMOTy o«^d pjrt*X)k)^. rc»H*f thon mon tft ^mr^t 
TVrwHroif u4 OwflOfitn, d«tB«Wd poit^*i o* 

>tfC^ t>p«» ei tfv flimv^. oppvcvd dum^ dtd t.m< 

81 DRP Units 

;#f*priO« » pre^e'«*nt« ^O* O^OfiO** »t<»f^ or»d f**! 
riJ#ofviot.on »r>d*p^ndeof fo«T^« ««^tet>fsi O 

Or«J r»fof»f«*if ptOipt^OUi. CCOtVX^ bo>* ♦© 
fi»fKtrf>' %Jitwi'^if H# prfC««w»d »^ fermf* Qi 

e^o^o*T^^<'7f^^f iodtp^f^de"*. o^d ttiw$ uA^iktHy to 

KtfteruS&f hi» t*>dfFnmt ot O C»*ta«n »o tt>« ^ftfi^|*i^c• 
dftmo^o^vci d^itin and dt$tru«s erf ciSfft dr- ^ed 

tfO<^ O fOfWKliOn »>«rt ^MtXm conditio*^. Wptf rt«f*T *0* 

t*« po<^*« cia*.»M, fo'jtd flym »ftto mch o b«#.f 
f<<^9Qi« it'ire* Mrf* pf«»*fvotwap t^«J tt»»t* no*«^o' 
1^ flf trns4! covid rtot fa* f »l«ed i»sMyi to prQ<)^f w. *o' 
ha'^H^v Of to ^MOtorctfi> f«VHyUfble Ciii^mt^'p 

So^ t^ •«o<KAr t»«>Je«'0^ 0^ ftoaM t<K»* te^if ta^VMi 



263 



100 1 



Rgurel 



70 



05 



60 



50 



45- 



57 {nform«(jtf0 



64 irtormnnig 



63 iwf>9 



58 Fiction. 



55 P^rsonalrty 



4a Fiction 



7? tmemjww) 



77 gxpeftmefitgt 



74 Appitea 



73 PeacTtpinre 



72 Anstyuctt 



70 Business 



fia Sc enes 



6/ UartyLtvtn9_ 



66 Emenatfmveot 



65 COT^unnH" 



64 HouM Edftortst 



63 Rj CPBStion 



62 PoQptft Sporti 



56 Fiction 



30 



T The Average ReadaNltty of SectiOTS of Prof esslonaf Journals 

) and High Oreuiation AduH, Teen and am Magazfaii^in DRP Units. 



264 



2. 



3. 
4. 



WHAT SHOULD BOARD WEMBETO KNOW ANDXOR DO ABOUT 

TESTING 

Uses of Testing: a evaluation of curricuhjm over district 



b. extent to which student has/has not mastered 
curriculum 

c reasons stu<tont has not mastered curriculum 

distr^or analysis 
d precfiction of future non-test performance 



Note: Remember: we are tea<^ing children information, skills, and 
how to reason: we are ncA teaching chiiren about tests, (unless the 
rest of the worfd makes us.) 

Testing has no automatic consequences that are not negative. 

Ooni g: . / a test unless it does a-c above, tn panicular, ask: when test 
results come in» do they tell us what we must change in teaching a 
particular chikl or in the curricufum. If no change can t>e inferred doni 
give the test. 

Using Tests: a bringing the task into the test versus bringing the test 



to the task: K-1 testing. 

b. achievement te^ng. Do they satisfy any of the uses 
given above. Stamiardized tests versus i.'itiondt 
norms, 

c. altematives to achievement testing: the problems of 
context, time and currfcukim relevancy. 

d. testify and state-federal funding, requirements. 

e. cunfcutum based tests: role of the state 

i competency tests: competency at Mtiat; who 
decides? 

g, predtotlva tests: are they worth usir^? 



Report Cards: a. school disti^cTs report card. CAR et al. 

b. the nation's report card. NAEP et al. 

Bias and the fear of failure: some human consequences of testing. 
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Chairman KiLD££. Thank you very much. 
Gordon Ambach? 

Mr. Ambach. Thank you very kindly, Mr, Chairman, Congress- 
man Petri, Congressman Hay^, and members of the staff. You 
have passed j^our own endurance test for this morning. Perform- 
ance test, I might note. You have been extremely patient and very 
attentive to the orientations that have been made. 

I speak this ahemoon, Mr. Chairman, as the Executive Director 
of the Council of Chief State School Officers, particularly today as 
the United States Representative to the General Assembly of IE A, 
which is the m^jor international education asse^ment organization 
that we have in the world. And because my good colleague has 
been making references to outcomes-based assessments in the State 
of New York, which I happened to put in place, I am going to take 
credit for having been a wew Yorker and the commissioner who 
was involved in that State, which does have^ as a matter of fact, 
the most extensive asse^ment system in the United Stat^, lK>th 
individual exams and performance or pr(^am asse^ments. 

Mr. Chairman, I commend you strongly for this hearing. You are 
smoking out the issues, and that is extremely important here. We 
have to be looking at what are the purposes of tests, what are dif- 
ferent types of tests, and I would submit that already you might 
reach these conclusions from what you have heard yesterday and 
today: 

One, there must be a unique United States solution to the issue 
of national or nationwide testing. We cannot simply borrow from 
somebody else what they have done because it has to match onto 
our own national organization, nationwide oi^anization for educa* 
tion and its unique cnaracteristics. 

The second thing I think is that the real issue before you is what 
kind of nationwide testing or assessment system we should have. It 
is not really the issue of whether there should be a single test or 
not a single test. It is much more complicated than that It is the 
question of how you put the pieces tc^ether in a nationwide assess- 
ment system. 

And it has also been clear I think from the presentations that we 
have oodles of testing and assessment at the local level, at the 
school district level, school levels, State level, national level, even 
at the international level. A critical issue, however, is how one con- 
nects up tho^ different systems, tests and systems, so that there is 
a common currency of information about the results, and that it is 
possible to be able to relate information about testing in Iowa or 
testing in Idaho with information about testing someplace else 
without necessarily having to test every single student in orde^ to 
be able to do that. 

Now, you requested, Mr. Chairman, that I speak particularly to 
the issue of the international dimension of national testing, and I 
would point out in simple form virtually every speaker that has 
preceded me has made the case for it, and the comments and ques- 
tions that have come from the members of the committee have 
made the case. Just think through this morning at how many 
points along the way someone has made a reference to a cross na- 
tional comparison of education, someone has made a reference to 
what rank ordering there is or made a reference to experiences in 
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one countnr as against another, and that is precisely the point that 
I bring before you. But, as you are thinking about what might be 
done with national testing or national asse^ment* keep in mind 
this international dimension. 

From both the understanding that we see and the misunder- 
standing that we have acro^ the country, it is absolutely critical 
for us to be investing more in careful analvsis and study of what is 
happening across the world. Not just in the scorecards, which are 
the popular public media items that demonstrate where the U.S. is 
relateo to other coimtriM, but in a much more profound sense than 
that. What do we learn about the type of instruction, the nature of 
content? Who is involved? Is there screening in the system by way 
of where students are placed? What kind of instructional approach- 
es are being given? What are the overall system characteristics? 
Those are the things we need to know. 

Now, for about 35 years, there have been a fair number of inter- 
national assessments, and the value of them has first and foremc^t 
been on trying to understand what are other systems like; and, sec- 
ondly, to try to get some kind of common formatting for being able 
to assess pupil achievement so that you could relate what the 
system is like to what the achievement is, and it might help to 
inform us as to whether we could change or make iK>licy for educa- 
tion in our country and other countri^ on the basis of that infor- 
mation. 

The fact is we have a worldwide natural laboratory of different 
educational approaches, and whether you are talking about the 
issue of vouchers or choice, or the organization of schools, or 240 
days or 160 days of schoohng, and so on, you can if you are careful 
about it, as my colleagues pointed out, extremely careful about it, 
undertake to look at what happens where those practice have 
been put in place in an attempt to try to get some sense of what 
the result has been. 

Dora it automatically lead you to a conclusion as to what we 
should do in the United States? Of course not. But it is a natural 
laboratory that must be used. That is the value of the international 
dimension. 

The second major point that I would like to make for you is that 
it is critical in doing international studies to have reliability, vplid- 
ity, fairness, an even playing field, if you will, as best you can, by 
way of making these studies. Within the last three years, there has 
been created in the United States a new board, which is the United 
States Board for Comparative International Studies. This document 
of framework and principles has been produced by that Board. It is 
under the National Academy of Sciences. I won't go into any of the 
principles or the framework, but it is an important guidepost for us 
all here in the States and in uther euuniries to be usitig in terms uf 
constructing these studies. 

The third point that I want to make has to do with what is np 
for this next decade. Already on the drawing boards, in fact, in 
progress and with the support of both the National Science Foun- 
dation and the Department of Education in this country, Canadi- 
ans with strong support is probably the largest single international 
study of education that has ever been done. It is called TIMSS, or 
the Third International Mathematics and Science Study. 

ERIC 
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In 1993-94 and in 1997-98 there will be a very extensive study of 
those subjects with three populations— nines, thirteens, and seven- 
teens— and done in such a fashion that there will probably be some 
40 nations that will participate. The emphasis is not g'^ing to be on 
just what is the achievement level, but rather on the study of the 
systems. In the jargon, what is now being called the opportunity to 
learn. What does a student have an opportunity to learn in each of 
these systems, and then what are the results? 

I would urge that even though this is not an authorizing issue 
that the members of this committee be attentive to the appropria- 
tions aspect of this in order that we have sufficient funds to carry 
that study through. 

And there are additional studies: in reading literacy, one in early 
childhood development and education, and a third one in the use of 
computers in sch(K>ls, which are under way in the lEA. So, at the 
end of the decade there will \^ a significant study in how it is that 
children in various countries learn a language other than their 
native language, something which I think could help to inform us 
in the Sta^^s greatly. 

My last and final point has to do with how it is that we directly 
connect these international dimensions with our national toting or 
nationwide assessment system. I had the privilege just a week ago 
of testifying on the Senate side, and 1 testified to the effect that 
what we need is not a national test, but what we need is a nation- 
wide testing system. Some aspects of it are what we call program 
asse^ment: sampling of students, a relatively low cost activity 
which tells you whether the system is pn^ucing; and you also need 
individual exams or tests which tell you whether individual chil- 
dren are learning toward particular goals. 

What we have to think about is how to conn^ those two differ- 
ent tjpes of tests in a national and in an international sense to 
common standards, and you hearf^ 'hat over and over this morning. 
Common standards against whicu we design tho^ tests, so that 
there can be a connection. And so think in terms of what frame- 
work we need, what apparatus. I happen to believe it is a national 
board for purposes of establishing these kinds of standards for our 
assessments as related to the international dimension as well as 
the national. 

One last point by way of these connections has to do with NAEP. 
The authority for State-by-State ends, unless you change it, in 
1992. Many of the States which I directly represent are extremely 
interested in thinking down track, to want to continue to have the 
State-by-State option, and they are also, perhaps to your surprise, 
thinking about wanting to do a subsample of the lEA TIMSS study 
so that the State might even be able to relate its results to the 
international study. 

The point here is that your early d^nsions on NAEP, the author 
ity for State4>y-State and the continuation of that, are extremely 
critical right now for the several States as they think about their 
own State assessment systems, how they are linked with the na- 
tional, and how they link with the potential international studies 
down track* 
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It 8 an immense agenda that you have before you, Mr. Chairman, 
and members of the committee, but it is an extraordinarily impor* 
tant one. It is one on which it requires the kind of care which you 
have heard expressed over and over again here this morning, and 
we would like in the future to help you in any way we possibly can 
to reach the right conclusions. 

Thank you very kindly. 

[The prepared statement of Gordon M. Ambach follows:] 
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Mr. Chairman, Members of the Suhcommiliec, and Staff Members of the 
Subcommittee. Thank you for the opportunity to testify at this tmponant heartng on 
"National TcMing: Ptos and Cons." Ai your request, f wiU fociM my retnarlis on the 
mtematKMUii uimei^ioas of national testing. ! speak as Executive Director the Counct! 
of C'href State School Offtocrs and as the Umted States Repfesentative to the General 
Assembly of the International Association for the Eva* .otion of Educalionat Achievement 
<IEA>« the most significam international organization for imemstional studies. 

I have four points to make. First, the vdue of international comparative studies, 
second, the necessity for a compfe!tenstve framework and principles for conducting 
international studies: ihiril^ the importance of United States* support for IE A studies 
p\annc6 through the decade of the 1 Ws: and fourth, the relationship between intemaimnal 
studies and nationwide assewnent in the L'mted States. 
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1. The Vklue of Inteniationai rnmpArative Educati on SmdtgiL The value of 
tntemational comparath^ education studies includes the following: a) they {Hovide us with 
an understanding of the differences in cuiricuJum, insma^iona) practices and system 
structures in the various nations; b) they provide measurement of student achievement in 
different nations in a format which makes comparisons of r^ts possible; and c) they help 
us to understand the reasons for the differences in student performance in the various 
nations. For the United States, with the recent establishment of national education goals, 
international studies have a new importance in measuring comparative prepress, particularly 
in mathematics and sdenoe, toward our goals. 

Up until the 1980s. nu>st students and parer*^ thought about student performance in 
comparison with their neighbors or, perfaar ^ smdents in other schools within the school 
district As one travels now across the United States talking with students and with parents, 
their coooem is about performance related not only to the neighborhood or community but 
to student performance in the state, the nation, and particularly to students in other parts 
of the world. Our students and their parents recognize their future is in an international 
society and their assodaticm with peers around the globe will depend upon the education 
they have compared to that of students in Sweden, la|^ $ing9{)ore. Germany, Nigeria or 
other couotriesu They recognize, furthermore, their future employment is in an economy 
which knows no national boundaries^ 

The common public notion of international studies of education is typically the view 
of a graph in the newspaper or on television which rank cmters countri^ according to 
an aggregate measure of student achievement on a particular math or science test The 
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assumption too often U that the only purpose of the comparison or the study has heen to 
see who Is wmning the "education race.* 

In fact, intematiOQal studies of education first began with the purpose of leamii^ 
more about the nature and type of education provided in the several oounth^ so that there 
could be an understanding of the varied opportunities to leam afforded students in different 
countries. In order to judge what rmlts yielded b&m that c^portunity to learn, 
achievement testing was coupled with these analyses of educational programs. Over a 
period of three decades, a scries of studies have reported both a description and analysis 
of educational programs in different countries and the student achievement results related 
to those programs. 

In the last 15 ^an» as we have become more and more ooncemed about the 
effectiveness of sdHM^ling in the United States, we have turned greater attention to the 
analysts of educational performance in other countries. The re^t of these studies has 
brou^t to light relative achievment levels of American students and, perhaps more 
important, drawn attention to significant differences in the characteristics of schooling, 
oi^ganization of schools, and the content of instruction in different countri^ The variety 
of approaches to education provide us with a 'naturaT wt^ld-wide laboratojy to examine 
different educational tediniqties and associated results of student achievement We need 
to expand our capacity for such international comparative ^udies so that we may leam more 
and more about practices in mher countries and the w^ in which they may inform 
policymaldi^ for education in the United States. 
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1 Frameworks and Principles for International Comparative g^4itt Intemalionai 
con^Tarative studies must be undertaken with validity, reliability and fairness. Although 
often difficult, it is extraordinarily important to assure examination of both practices and 
results is done with Refill deAgn for good sampling of student populattoo, valid and 
reliable testing, and so that there are fair comparisons made where there is considerable 
variation in instruction or the curriculum* Three years agio througb the support of the U.S. 
Department of Education and the National Science Foundaiioii, the Nai.'onal Academy of 
Sciences, National Research CcHinctl formed a new board, the Unit'^d States Board for 
International Comparative Education Studies (BICES), for the |. . pose of providing 
guidance to United States funding ^encies and to researchen in th^ US. and across the 
globe on tbe conduct of international educat.on studies. The Board has issued an important 
report^ *A Framework and Principles for International Comparative Studies in Education.* 
attached to my testimony. There is not time here to review the report, but I would urge 
your consideration of it This report has been ertremely well received both in the U.S. and 
abroad It provides the groundruJes for United States' participation in studies in a manner 
to assure the funding agencies. Members of the Congress and the public of credible results 
from tovestnKfits in studies. 

3. Support for lEA International Comparative Stui^ies drying thg Deca de of th^ 
lEA studies have been undertaken for 2S years. The lEA has launched the largest single 
international study of education ever conducted In 199^94 and again in 1997-98, there 
be a major study of mathematics and sciences, the Third International Mathematics and 
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Sdeoce Study (T1MSS). Ncarty 40 nations will p&nictpate in this study with sample 
populations of 9, 13, and 17 yc^ olds being tested Extenuve reviews of their "oppommities 
to iearQ* - the curricuJum, instntctitmal and school patterns in the different countries - will 
accompany the testing. The United States Department of Education, Natioiml Onter for 
EducatioQal Statistics, and the National Science FoundadcHi have already made significant 
commitmeots for the implementation of this study. Design work is proceedii^ under 
guidance of the lEA headquarters at The Hague and the International Coordinating Center 
for the study at the University of British Columbia in Vancouver. The design of this study 
has been significantly ioOucnced by the UoiteU States in order that the study will yield 
significam information on ^ucational programs ami results related to progress on National 
Coal #4, Mathematics and Sdence. 

lEA has other studio underway* induding those on earfy childhood development and 
education* directed by David Weikart of High Scope Educati<Hial Research Foundation tn 
Y{»ilaflti, Midugaa Ttere are also studies of reading literacy and of tte use of computers 
in education. Still other studies will be launched at th^ Matter part of the decade, probably 
one of Ic' jnii^ languages other than a native langimge <rf eadi countiy. 

i urge your moog support for financing these international oomparative studies. 
The general authority of the U.& Department of Education and the National Science 
Foundation is in pbee; the is providiog r^ources timefy for fn-eparation of state of the 
art assessments^ an eff^tive cn>ss*oatH>nal ooq^eratton needed for ioptementation aru] 
analysis of the results. 
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In tematiQnal Studies Relateri to Naiinnal T>«^ng The focus of the hearing 
today is on national testing. The design and conduct of intemationaJ studies is an important 
part of discussions of national testing. A major chaUenge before aU of us is in making better 
sense of the various testing and assessment systems at all levels - local state, national and 
international -- to make them efficient and to provide that results at any level may be 
related to resulti at other levels. 

Mk . Chairman and Members of the Subcommittee, a week ago I had the opportunity 
to testify on the topic "Question of a National Test* before the Senate Subcommittee on 
Education. Arts and Humanities. In my statement before that subcommittee, which -s 
attached to this statement, we recommended the answer to the quesUon be a nationwide 
testing system. My comm-nts today on international studies are made in the context of our 
proposal. 

International comparisons in education have been and can continue to be based on 
assessments which sample student achievement. In international smdies we are interested 
in knowing how entire systems or unjts of education perform rather than how each 
individual student performs. However, through international smdies, we want to have a 
means by which achievement levels of studems in any part of our country through individual 
examinations can be related to standards and results which are international in <icf)pe 
International comparisons must be related to nauonal standards which in turn are the 
gutdeposts for student performance in any part of tbe United States. Through the link <,i 
nauonal standards related to international assessment, we can determine the rc!;;tive 
performance of individuals in the United States to that of smdcnts in other ccumries. ITic 
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key point here* is the impoitaDce of having a process and entity for determining national 
standards for student testing. 

1 wili not du^II on other features of the nation*wide testing system as described in 
the attached testimony, but I must emphasize the importance of constructing our national 
system so that there is an efiBcient connection with our national results to cn^ national or 
international comparative studies. This has not hai^sesed in the past It will be a bold 
tnove for us to plan and implement a system which makes such relationships possible. But 
is is imperative that we design such an approach. 

To expedite the participation of the United States in international studies, it is 
essential to plan for developn^t of a nationwide testing system related to international 
studies as they may be used by the several stat^ Many states will prol»bly participate in 
the 1993-94 Third Interoatimai Mathematics and Sdeoee Study (TIMSS) just as the 
Canadian provinces will participate in addition to Canada as a whole. The states need p{an 
carehiUy now for 1994 with r^pect to intentions in participadng in NAEP* as well as in the 
intemationai study. There should be a close retetionsh^> between NAEP and the 
international study and, therefore, we hope there will be early authorization to continue 
state [^rticipatioo in NAEP. I urge Urn oommittee to take acdon to continue authority for 
the stat» to use NAEP on a state-by-state basis in 1994 and beyond so those states which 
do choose to participate in NAEP and also hope to participate in TIMSS m^ design their 
overall state assessment plans accordii^. 

In condusioQ, Mr. rhft"'^^", I note that studies done through lEA in the 1960&, 7Qs, 
and early 198Qs. yielded some of the most important information for alerting the United 
States to the necessity for major reform and restructuring of edueatioa Reports such as the 
Underachievi ng rSirrigiihim! A«i^>sing U fL Mathemadg^ from an international Perspective 
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which cime fran the second imeniaiioaal $ni<ly of otathem^cs was very powerful in 
demonsiraimg the reUiivefy poor perfbnnance of Americao students its maihenuuics. Tliese 
uudics have oot just displayed ftchievemcm results, but they have presented {juportw 
information about the time cummiited to instn^oa the imture of the ittstruction, 
commitmeacs for homeworie. commitments lo staolards expected of students which have 
sent powerful mes&ages to consider here in the United States We muM inoease the (low 
of information from &uch studies. Wc cannot tse insnlaf nor provir^iat in our approach to 
suengthening American education. One of the most imp<»tant resources of infonnaiioo for 
reform here is in ».hc examination of educatton in other countries. 

Chairman, Members of the Subcommittee, thank you for the opportunity to appear 
before you this morning. I wiJI be very pleased to respond to any questions about my 
remarks. Thanit you. 
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MARCH 7, 1991 

VS. aCNATS SUBCCMMITTCE ON EDUCATION, ARTS AND fa^fAMTIES 

L Chairman Pell members of the Senate Subcommit^ and Staff of the 

Subcommittee, tbank you for this {^^portunity to respond to y<»ir uxvitatioa to testify on 
t>vestion of a Naii(»iai Test* At the outset, Mr. C^sairmao, I recognize ond commend yoa 
and your colleagues who have been advocates for national examinations long before serious 
consideration %vas beii^ given to these is»ies for most of the nation. I m>te e^daily your 
respoQSibiliiy for authoring tte provmon for national examinations in the Hawkins-Stafford 
An of 1988 and commeiul yon and numbers of the ^ibcommittee now for bringing the 
issues before the Senate in the manner of this hearing. 

Testing is at the center of learning. The diaractertstio and manner of 
questioning or inquity guides the form of education. How tests are constructed shapes what 
is thought and l^raed and* therefore, decisions almit testing roust be carefully made. That 
is especially true now because of fl» relation of testing to reform and improvement in 
American education and the location of education decision-making in the Uoiied States. 
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Our Amencan partem of decenrralizatioo of authority for education in states and locaUtie$ 
is being weighed off against important national goals and purposes for education. Who 
estabiisbo^ standards and develops tests is« in many ways, as signiScaoi a question as the 
content of the test The right combination of common standards with variations of tests 
which tneastire progress toward those standards is central to a national solution. 

At the very time we search for successful refcMins and improvements in 
education and debate the relative levels of decision*making at different points in the 
educational structure, we must also adjust to rapidly changing techniques of assessment 
While we have a suong need to do things nationally - to improve opportunities for students 
nationally and take steps to increase our national competence through education - we must 
be certain ;o enable and encourage variation. ejq>erimentation and innovation that enables 
us to create acd reaeate ever better systems of testing and learning in the future while we 
are putting new standards and tests in place now. 

2. Our response to a "Question of a National Test" is to recommend "A 

Nationwide Testing System" which has the following three key elements: 

a) A procedure and national entity for setting national standards for student 
performance, subject by subject 

b) A system of both propam assessments (through sampling of student 
performance) at nauonai, state and local levels to detertnine program 
e^ectiveness and a system of individual ewninatiors which might be 
nationwide, multi-state, ^tate or local in administration which measure 
individual stjdent prc^ess on the national standards. 
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c) A reporting system whkb enables studcpts, parents and responstble educatioo 
ofiBctab anywhere in ihe United States to be able to relate any one stuctent's 
achievemem to tbe national stamianis and tltt perfonnaim of otter students 
in the cotnunitnity, statet nation, and even the world* 

3. The gist i letnent is to establish a pnmdure and an entity for setting national 

standards to tte nationwide teeing ^em. The United States currently has no naaooal 
entity to establish national uudou perfbrmanoe stamkrdi Such an entity, ih* Boards nnot 
be carefully desig2»d sod establistoS thrcnigh an Act of the United States Congress and the 
PresidMu. The Board should be comprised ol distinguished persons appointed in equal 
numben by the Cr4gre$s ami the President Appointments to the Board stould be based 
on a thormigb nominatmn process n^ikh assures Board members wiD be weil^^piaUfied for 
their respoosibiUty. T)» Boanfs responsibility stouU be to establisb 'trameworks' of 
studcm pexformanoe goals and objectives, or standardSi upon which both program 
assessments and imihUoal studM naminatfons are btted Tbs proctsA of setting such 
frunKworks most im^olvs ifiey education authorities at state and local levels* 

Ahbwgh it might seem estabUshmem of such imtiofial fra mew o rks is foreign 
to A p» ri c an education practioe, the fact is that sncii a prooeci h hi place to tte National 
Assessment of Educational Progress (NAEP). The process has already been used for the 
subjects of natl»matics^ readii^ and the scienoes under the direction of the Gmndl of 
CJiiefStatcSchodOflBeeiiasapartof thedevekymentof NAEP etaminations. Astmilar 
pn?cf« could be used by the new Board 

A seomHl iiqxiitam reapomibility of the Board sbo^M be review of various 
proposed program assessments and individual examinations in order to determine whether 
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they effectiveiy measure the standards ^tablisbcd by the Board with reliability and validity. 
The Board would maintain quality controi of various tests to assure rigorous measurement 
of both what students know and can da Hie Board would exerdse judgement on 
efTectiveness of a proposed test for its intended purpose and the de^'gn for reporting results 
on the test. 

At this time, Mr. Chairman, we are not presenting a spedfic bill for the 
creation of the Board, but we would be pleased to assist you and members of ihe 
Subcommittee and the Staff in the development of such a bill 

4. Program assessment components of the nationwide testing system might be at 

the national, state and/or local levels, NAEP provides the basis for the major program 
assessment component For nearly a quarter century, NAEP has been providing periiuiic 
testing of samples of students across the natioa It provides overall trendlines for student 
achieven^nt 

In 19S& Congress authorized the use of NAEP on a state by state bas^s. In 
1990 the Urst use of NAEP on a state by state basts - mathematics at the eighth grade tevei 
- was in^lememed You have authorized state by state NAEP in mathematics at tuo 
grade levels and reading at one grade level In 199Z Authority for state by stat^ NAtP. 
however, stt^ at that point 

We urge you take action as rapidly as possible to authorize the continuauon 
of NAEP on both a national and state by slate basis in five major subject areas 
mathematics, science, readii^ writing and history/geography - each to be tested evcrv ruo 
years. We urge, furthermore, that you authorize voluntary participation in NAEP at a 
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school district-wide level for tbc^ districts of sufBdeot size for appropriate N AEP sampling. 
We have a marked up bill which would accomplish the ameiuiments recommeiided above. 

Within the overall natiwiwidc testing system^ NAEP tests would be constniacd 
lo measure the subject objectives and standards established through the new Board 
frameworks described umler point (3) above. 

5. A nationwide individual eiamination system should be established wit^in the 

nationwide testing system. This system could have several differem forns of ex^uninatiom. 
These might be used on a nationwide basis or by dusters of states or disoicts» or individual 
states ov school districdw The detemunation oi their use wiHild be made by state or local 
education authorities. Tl» examinations would be based cm standards establisbed by the 
Board umter item (1) above. The types and form! of varicRa OKamiBBHons would differ, but. 
as noted above, to be pan of the nationwide system they wcmid have to be ju(^d 
apprqiriate to medHtre the national standards by the Board. This system would enable 
creatioB ol inMvative forms of tesdi«, iiidu<&% peifcroiaaGe assessment, and enable a 
variety ^jproadm by states »ad localities in esttbli^liing individual ex a minati ons in an 
efficient and cost^enective nmnner Ttoe ezamis^diom mtist be dosely associated with the 
nfrWrithtm; iit A variety of patterns at different grade levels according to those points 
study at which subject mastery is conq^leted; a«l be used as "high stak«' tests related to 
credit or credentials for txuSividual students. 

States ami tocaiities t^ve a varied of emtiog individuai exBmination systems. 
il» most compi thcnsiw now being the New York State Re^ts ettminations. Different 
systems, such as that in New York, could ccrporaicd into the nationwide individual 
eaamination systems^ 
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Of special note, Mr. Chairman, is the potential iocorporation of tl^ voluntary 
national examinatk>o auiborized in the Hawkins-Stafford Act of 1988 into this ^tent This 
examination has never heen implemented It could be one pan of the naticmwide individual 
examination sy^m, its content guid^ by the standards ol a new national Board 

We recommend you take action to support the research and development 
neected to establish a nationwide individual examinadoa system, 

6, Common and consistent reporting of results ^m the different program 

assessments and individual examinations is of central inq)ortance in the nationwide testing 
system. There is km)w extensive testing at all gracte levels for American students* The 
information from the tests* however, generally cannot be related so that results from 
one sctool or schodi distria may be compared with another district within the state, or 
outside of the state* «r outside of the nation. One purpose of the nationwide testing system 
is to create the means for relatugig results throughout the system without the necessity of 
requiring ail students at all grade levels in all subjects to be tested on the same tests. 
Reporting systems must be established nationwide with ittfcHination on studem results 
related to other education indicators. This enables better understanding of the causes of 
student success Of failure and helps results of testing lead to program and student 
improvement 
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7 Tcsxiog sy«cm* carry ^igmficani costs la iWdcnt insmictioaai time, teacher 

ttme. and in the cwt of creauon, adountttraisoo Koring ami reporting of test results. 
Careful estimates must be made on the trade ofT& between tKOgraro a&sessmcm sanipltng 
versus individual etaminations and 'evtry student testing," Careful estimates of cost must 
he made over ih s decade so that an cffiaent mix of program assessments and indmduai 
examinatjons > - created and coordinated m) as to Umit ci^is locaUy, state by state, and 
Federal. 

A nationwide lestmg sy%icm is essemiai. Information about the nation's 
education status ts certaxnly as important as information abc«it ihe naiion*s heahh. its 
agnculturai coftdiiton. and the condition of labor and f n^Joyment In each of these other 
areas, the Federal govcmmem is now spendii^ approwmaiely $a times as much for the 
coileaion of informauon about performance and system indicators as » trve in education 
11>c commitment to a i^oDwide tesung system must he accompanied by a commitment to 
a Federal budget which makes certain tbe lestii^ system ts of as high quality as we crpeci 
Htudent pcrfonnancc to he. 

M/. CbainnaD and Members of the Subcommittee, thank you for ^he 
«>pporrunjiy to testify 1 will be pleased to respond to any questions. 
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Chairman Kiij)E£. Thank you very much, Gordon. I appreciate 
your testimony. 

Tm going to start off with a general question, one that I have 
asked all the panels. I will shorten it up a bit. 

Can we make testing, specifically national testing, helpful in the 
improvement of education? 

Dr. CooPERMAN. I will give two examples, and theyVe both hope- 
fully connected. 

One is if we find out that by t^ing all the children— and I sure 
agree with Gordon, and I really agree with Governor Romer and 
Mr. Brock— that you really have got to know where you're going. 
You have got to know what it is you want to teach, and then have 
the standards. 

liet^s assume that most of the teachers in a particular State come 
from four, five, six or seven graduate schools of education. Let s 
further assume that the children aren't doing very well in math in 
fourth grade. In fact, thevVe doing kind of poorly. That would be a 
powerful messfi^e to go oack and sa^ vou were not training your 
teachers correctly^ because if the children aren't learning, maybe 
they're not being taught. 

het me give you a second practical situation, I asked the question 
why, in mathematics, are New Jersey s children and the children 
of the United Stat^ in elementary schools doing so poorly on every 
international comparison I could find? The answers were not diffi- 
cult to find out. of the teachers had never taken a math 
course; most of the teachers — I hope this doesn't come out as sexist; 
I don't mean for it to — but most of the teachers in grades were 
women. They were told they were not good at math and so were 
not math m^ors. 

The textbook manufacturers told me '*we're not going to have a 
faster math pace because the teachers can't teach it." We taught in 
New Jersey 28 minutes of math — we conducted a study to find this 
out. We taught less math than anything eljse because the teachers 
didn't want to teach it. 

Well, let's say that the results on a national achievement exam, 
or clusters of exams calibrated, that they show we're not doing well 
in math, specific and precise. Mayte what would happen is teach- 
ers would now take mathematics courses, that the school day 
would be extended, and more time would be allocated for math. 
There would be staff development or in-service training, and the 
curriculum would be more aggressive. 

I tried something like that in New Jersey. I went all up and 
down the State and said we're not doing well. We're not doing well. 
People iust turned away and said, ''What are you getting so excited 
about?* Our graduate schools were mainly taught by Asians. The 
majority of the kids in mathematics graduate education were 
Asians. And yet there didn't seem to be much of a concern that 
we*re not doing well 

Yes, I agree with Monty, that we have got to watch out, that 
we've got to have good t^ts and fair testa, and we can't abuse kids. 
We have got to do it right. I think we do have the ability to do 
that. 

When the results come out, this will drive the ingenuity of 
people to figure out how to get things better. Right now there is a 
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malaise. People will bait and hook you and try and get you talking 
about other things that we ought to do because they don't want the 
data. 

ril give you two examples. The data will go back to the teacher 
training and make that more powerful* and it will release a local 
ingenuity to figure out how can we do better once people are con- 
fronted and once they finally face the fact that we are not doing 
very well. We have got to face that fact first. 

Chairman Kildee. Dr* NeilL 

Dr. Neiix. Yes, thank you. 

Let me start by answering with a very simple example. If Maria 
can't multiply, why not? Well, the first one is let's araume she 
hasn't been taught. We don't need national exams for a teacher to 
know whether or not the teacher has taught Maria to multiply. So 
that's not helpful. 

Let 8 say the teacher has taught Maria multiplication but it 
didn't catch. Why not? The current kinds of exams that are used in 
multiple choice tests provide no useful information on that topic. 

The best thing to do, to start with, is to look at the kind of rea- 
soning that Maria is doing mathematically. Here we need, of 
course, a teacher who is educated to be able to, one, know the kinds 
of questions to ask the student — and we don't do that a lot in our 
schools — and, two, to interpret that information into revising the 
kind of education material pr^ented to Maria, the opportunities to 
Maria. It's not just the teacher talking but the different kinds of 
math experiences for Maria to go through in order to be able to 
understand finally how to multiply. 

Testing is not a gocxl mechanism for providing feedback for in- 
structors. The kind of information that Mr. Cooperman was talking 
about can be provided by light sampling on the order that NAEP 
does. You do not need to test every child to find out, across the 
system, what kinds of things are going on. 

The reason, in large part, to reform the kinds of assessment that 
we do is because performance^based kinds of assessments simulta- 
neously involve instruction and assessment prwesses. But they are, 
as I said earlier, extremely difficult. We don't yet know how to 
turn those things into a nationally aggregated system. 

So we then have a quandry. We go down the road of simplistic 
tests and that kills the education reform process, or we begin to do 
the more complex task, knowing full well that we don't have an 
outcome in sight. So the short answer is that most testing cannot 
help us in our education reform. Carefully done, some kinds of as- 
sessment can be used, provided that we have enough information 
surrounding it to make sense of the particular test information. 

Chairman Kildee. Dr. Kelley? 

Dr. Kelley. Yes. First 1 would like to {K)int out that we don't use 
what*8 available with respect to current testing to help the teacher, 
as J mentioned earlier. 

A second point is that all the problems mentioned that might be 
addressed by tests we know alK>ut, based on all the testing we do. 
We don't need another test to know that we have problems. Again, 
it's important to find out which of these kids don't know math, not 
what the average is. So I don't see another national test, particu- 
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larly if we can define the problems carefully now — and I believe we 
can — of use in that dimension at all. 

I do see some rationale for light sampling in terms of interna* 
tional comi»risons. It's in a wholly different ar^. That is always 
the Droblem of educational finance. We have ^d that educating 
the 30 percent of kids who are not suitable for the modem work 
force, directly or indirectly, is going to cc^ money if we're going to 
maintain other values that we cherish in our society and which dif- 
ferentiate us from other societies largely. That money is hard to 
come by, and it's particularlv hard to come by locally, where only 
30 percent of families have kids in i^hool. Any kinds of ammuni- 
tion that can be ui^ed locally to provide the Thomas Jefferson"' 
effect, as I would call it — that is, education is good for democracy, 
and we're not doinp; well compared to other democracy tyro argu- 
ments with local citizens— is always good information to nave on 
hand. But I see some value in international comparisons then for 
local financing of education, as long as it remains there. 

Chairman Iuldee. Yes, Dr. Hertz. 

Dr. Hertz. I would just like to suggest that the further the test- 
ing is removed from this opportunity for it to feed immediately into 
the educational process, the less likely it is to be effective. 

And then just one little side story. We may all want to watch 
Wisconsin with some inters over the next few years. Ten years 
ago in Wisconsin, a younger did not have to take the ACT or the 
SAT or any other test in order to enter the university system, one 
nationally accepted as being absolutely excellent. Children didn't 
have to take a test to get in. There were no hi^h school ^duation 
requirements in Wisconsin. The local school district decided what- 
ever it was that it liked for the kids to be doing. And yet, the chil- 
dren in that State have been doing beautifully, have been doing 
well, and there is an excellent work force there. 

Now, in the past ten years we have changed all of that. We have 
built rather rigid standards into the expectations of the State, we 
have statewide graduation re<^uireroents, and we now are forcing 
children to take these tests m order to go into the university 
S3rstem, I think we all should look to Wisconsin with a great sense 
of history, and as historians we are going to want to find out 
whether we get better for all of that. 

Chairman Kildee. Gordon? 

Mr. Ambach. Mr Chairman, I think your question takes us right 
back to the heart of the issue, and that is why it is that we need to 
be talking about nationwide assessment or testing systems rather 
than a single test. 

I have to split the answer to your question in two parts — one, 
whether we're talking about national individual examinations of 
youngsters, or whether we're talking about national program as- 
se^ments, which are the samplings. 

As far as individual youngsters are concerned, any value of pro- 
gram ass^ments has to be indirect, if you will. Program assess- 
ments are designed to try to help us with overall policy, overall di- 
rection of what the program should be. The impact on a youngster 
is not going to immediata but it is going to be in terms of how 
you use that information to inform a change in the program. So 
youVe going to have that as an indirect, That s as very important 
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function, though, at the national or State level by way of having 
that indirect impact on youngsters. 

The key impact on youngsters, of course, is with individual ex- 
aminations. But you have neard over and over this morning the 
difficult!^ in trying to pack it all into one single test. That is why 
our challenge, I believe, is to be figuring out the way that we can 
have systems through which there are many or several different 
kinds of individual examinations which get calibrated back to a 
common standard, that leaves you the flexibility of a direct effect 
for individual students in that particular examination, but gives 
you the advantage and, when the testing is done, there's a way in 
which you can connect the result to some kind of common stand- 
ard, so that you know what s happened. 

Greg Anng was talking about what happens when the teacher 
closes the door, and that's correct. The concern is whether the re- 
sults of what's happening when the teacher closes the door are the 
results we're looking for across that school, across that district, or 
in a broader scale. Tliat's why you need to have some way of being 
able to judge whether that individual performance is up to or close 
to a particular standard. 

Chairman Kildee. Dr. Cooperman, you had an addendum? 

Dr. Cooperman. Thank you, sir. 

What I have heard a lot of today, I guess, from the three gentle- 
men to my right, is frequently another test — my god, here comes 
another test — and therefore the test is painted to you that the 
whole idea of examination is bad and you should resist. Also an ar- 
gument is put forth that we already have the test. 

The tests that we have nationwide are basically three types: one 
is the Iowa Metropolitan or Stanfere. These tests are usually given 
to children in grades K-8. These are not secure tests, they're not 
called a high stakes test. The teachers frequently have them and 
coach the kids on them. And as John Cannell said on Lake Woebe- 
gone, all the kids are above average. He exposed that one. So first 
of all we have the Iowa-type test, which are basic skills in grade 
8. 

The second thing we have is State testing, which is usually in 
high school, and is only basic skills. The third type of test is kids 
who go to collie, ETC or ACT. We have no tests for all of our chil- 
dren that match and link~ril use the word link— to our national 
goals. What's different now, and why weVe talking atK)ut this now, 
thank God, in the history of our country, is we finally have nation- 
al goaJs, not State goals but national goals — history, math, reading 
ana writing. 

I just think that a fragmented system which has three sets of 
rather inadequate tests can, through this discussion, be combined 
into a better system. Our way says, if you don't drive it back to 
each individual school— I agree with NAEP, I agree with what 
Gordon says, and I agree with what Governor Romer and former 
Secretary Brock said. You try and find out the answer to two basic 
qu^tions: what is it you want children to learn, question number 
one, and question number two, how do you know that theyVe 
learned it. 

My concern is, if we just do sampling, we will know how the 
country is doing. We will not know how that school is doing. The 
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schooU Tm afraid, in many areas will get off the hook and eay the 
country might not be doing very well but Vm doing flne« 1 would 
ask you to please consider that. 
Chairman Kildee. Mr. Petri. 

Mr. Petri. Tliis has been a fascinating panel, as well as the ilrst 
one, and Vm very happy to have the opportunity to listen to all of 
you. 

I have a lot of qu^tions, but let me just ask one. If as a rrault of 
going to a national test on history, government, math, and so on 
and so forth American kids know more but know how to learn less 
well, will we have gained or Irat? 

In other words, is it all right, on average, to have American kids 
who don't know much in terms of specifics but know how to adapt 
and to learn better than, say, German kids or Japanese kids? That 
probably is the case on average, the way they perform when 
they're in akinies and given a random task to do, to go out and do 
it and show some initiative. There is a lot of self-initiative in the 
American people that you don't see in othei populations. 

If we sacrifice some of that in order to do a better job in terms of 
imparting specific data at a particular point in time, have we 
gained or lost, or can we have both? 

Dr. Neill. If I may go first, sir, we can have both. In fact, what 
tends to happen in this country, the CATS in metropolitan areas 
are extremely high stakf»s tests, and in many school districts their 

E>reimration occupies half of every school year. They are reducing 
earning to discreet little bits and kids don't learn the adaptable 
processes. 

Let me give you an example. In California a study was done of 
the very best students in calculus who were high school seniors. 
What they found out in this study was that these students really 
do not know how to problem solve. They can apply a rule very ac- 
curately, when they know what rule to apply, but they don't know 
how to approach a problem in order to define the problem and then 
go about the process of solving it. 

That's why I used my example earlier, that we need our students 
in history to think like historians. An historian will know the dates 
and the facts, but they will make meaning of the dates and the 
facts. If you just absorb dat^ and facts, you'll take one of those 
quick tests they give you three years a^r you leave school and, 
gue» what; you don't remember anything. The Soviet kids don't re- 
member anything, either. They got taught probably pretty much 
the same way. 

If you use information and can think historically, you may not 
even remember all the facts— you probably won't. But the point is 
you will know the processes. luiowing information is not antithetic 
cal to the deep thmking* In fact» you have to think about some- 
thing. But when we talk about lading, it's a matter of approach- 
ing children in the way humans actually learn, which is you adapt 
new information into the mental constructs you already have. It's 
a thinking procew. You do drills and it makes sense to get better 
at something, but you put it in its proper pla<^. We nmd assess- 
ment systems, not te^ that encourage thinking. If we have t^sts 
that don't do that, we won't get there. 
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Chairman Kildee. If the gentleman would yield just a moment. I 
taught school for ten years and a lot of times we test for the 
*'what" and we don't go for the "why." 

1 used to draw on the blackboard the opening day of each class a 
triangle and ask Uiem what the formula was to discover the area of 
that triangle. M<»t all the students would say one-half the ba^ 
times the height. Then I would ask why is that the formula. It's 
amazing that mMt people didn't know why. When you complete* it 
into a rectangle, you can empirically measure it, right? We have a 
formula when it's a rectangle. But I was amazed even then, that 
maybe in our testing we have to be careful that we don't just lead 
to the "what" and not to the "why." 

Thank you for yielding. You may have some additional time. 

Mr. Petri. Dr. Kelley. 

Dr. Kelley. In rwponse to the question, I think the extreme case 
you're talking about is the entering college exam in Japan. You 
look at where the Japanese send their engineering students when 
they actually want them to learn how to think, it's to American 
engineering schools. But more imix)rtantly, I don't think it*8 going 
to happen in the United States because, as contrasted to. say, when 
my parents went to school, if you have ever sat through a lot of 
classrooms, teaching of young children in particular is very much 
both and art and science now. These are pMole doing things that I 
don't know how to do, and I might be too ola to learn how to do it. 

Now, these people— there are a million of them— many of them 
have talked to each other in many professional associations now 
for years. It is wholly different from right after World War IL They 
have an idea about what th^se competencies should be, an they pay 
much more attention to that increasingly than they do to mini- 
mum competency tests and things like that. We are now dealing 
with people who actually should be here answering your questions, 
on the whole. In fact, it is somewhat amazing that I would be offer- 
ing answers to your questions about this when people who actually 
think about this every day of their life, and think with their peers 
about that, would actually know much more alx)Ut it, I think, and 
know much more about what is actually going on. I don't think 
they would allow it to happen. 

Mr. Petrl I did have another question of my fellow 
"Badger" 

Chairman Kildee. Go ahead. 1 used part of your time. 

Mr. Petri. I think the fact that here in Washington, although 
we're a smaller than average size State, there are more members 
on this panel from Wisconsin than any other State is just one sign 
of the value that our constituents place on education. I think we 
reflect that. We look in our Stat^ to education and value it very 
highly. It is what helps us to compete and do well. We're not per- 
fect at all, but we try to work very hard at it, I think, on average 
in Wisconsin, We do devote a lot of resources at the State and local 
level to education, to put resources behind every student, regard- 
less of the financial well-being of that community in our State, 

The question I had was, I wonder if you would expand on some- 
thing you mentioned almost in passing. You were thinking that 
mayte there was some hidden agenda with a national testing 
movement having to do with pushing choice or undermining the 
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public school system, something like that. I wonder if you could 
spell out your feare or concerns, what it was you had in mind? 

I>r. Hektz. It may merely be a reflection of an administrative in- 
stinct towards suspicion. Congressman. But as I was ^ing through 
this last year, and I felt the enormous number of pohtical foreh- 
and I don't just mean State i^^lators on our Crovemor's cx>mmis- 
sion, but from the unions £md special interest groups of everv kind, 
both of busine^ and from the homes, the school people, all ihe way 
across the spectrum, Impacting upon education, in a sense batter- 
ing it from one side to the other. As I came away from that, I think 
I came away from it with a certain wnse of suspicion about do all 
of these people, who seldom ever are in classrooms with children, 
are seldom ever in schools, are they all of good heart, or are they 
selling us something, or are they protecting something in terms of 
everything from jote to whatever. That raav be what you feel. 

I don't know that I can point to anything specincally linking 
choice to the test question, out I have to say it s in my mind. 1 
think that sense of suspicion negatively im^cts— all of these 
things pounding away at public education I think does have a nega- 
tive impact on us. I do suspect that in some cases the people that 
want a piece of the action— I don't even want to say that beat us 
down— but who want a piece of the action, want the schools to 
solve the social problems and all of the sort of things that are 
there, Tm not sure they're all of good will. I know many of them 
are, but Vm not sure that they all are. They all have the best inter- 
ests of a broad cross section of children at heart. 

Dr. Neill. Could I give a specific example and answer? In the 
Wall Street Journal the other day one of the arguments in favor of 
a national test advanced by Checker Finn was in order to have in- 
formation, in order to allow school choice to proceed, and he a^ 
elsewhere has favored eliminating local school districts. So again, 
that fits into it. 

I would just add that if we have a bad test, that will further dis- 
courage American education and further damage the possibility of 
enhancing public education. 

Chairman Kildee. Mr. Hayes. 

Mr. Hayes. Mr. Chairman, this has been a very, very interesting 
hearing here today. I could have several questions, but embellish- 
ing on what you said. Director Ambach, there is a limit that that 
part of the human anatomy which you sit on can endure. 

[Laughter.] 

We've been here now three hours and 45 minutes, so I just v ^nt 
to direct one question on something that bothered roe a httle bit. I 
don*t know if any one of you can answer it. 

We are facing a very critical situation in our educational S3rstem, 
what appears to be budding to the point where by the end of this 
century we're going to really face a shortage of Afncan-American 
teachers. Do you have any su^estions as to how we can tackle this 
problem? There seems to be little or no desire for the kids in school 
to become teachers, particularly among African-Americans. Do you 
have any ideas on how we can tackle that problem? 

Mr. Ambach. Congressman Hayes, Mr. Chairman, let me make a 
comment about one very important step that can and must be 
taken here at the Federal level. 
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Last year, toward the end of the session, your committee report- 
ed out the Excellence and Equity in Education Act. The House of 
Representatives passed it not only onw but tvnce, with some revi- 
sions. It went over to the Senate and was killed by a few Republi- 
can Senators at the end on procedural grounds. 

That particular Act carried, if my recollection is correct, more 
than $350 million for purposes of recruiting and preparing teachers 
for elementary end secondary schools and, in several respects, had 
a special emphasis on the necessity for the recruitment and prepa- 
ration of Afro-American teachers, of Hispanic-American teachers. 
You have cited, quite properly, a very critical problem for the edu- 
cational system. 

I have said repeatedly since that event, which I believe was last 
October, that the first order of business for this ^ssion of the Con- 
gress should be to bring back that goals bill, bring back that Excel- 
lence and Equity Act, which the administration had agreed to fi- 
nally, even after they had been here and telling you there wasn't 
any teacher shortage. They agreed to that. That bill ought to be 
put back in there in the fashion that it was before, perhaps with 
some alterations, and moved through as a significant act in order 
to address the issue that you^re raising. 

Dr. CooPERMAN. I have two ways. One is short term. You might 
check the alternate route to certification in New Jersey. I have a 
bias, like Gordon do^s, for the regents and so on in New York. In 
the past four years — and there's plenty of data on this— 23 percent 
of the teachers in the alternate route to certification in New Jersey 
are minority. They have come from some of the more prestigious 
colleges in the United States. 

I have met with the teachers, met with them for three straight 
years, and I asked them why do you want to teach in this State 
when you could have alm(^t any job open to you? There were two 
responses. One, '*! did not want to take \iO to 40 credits of teacher 
training in pedagc^." The second reason was, "I want to pay back 
and I want to make it tetter for some other kids.** 

Now, the average that are coming from the teacher training in- 
stitutions across the United States is alwut 5 to SVi percent. The 
prc^am has been there for four years and I think there's enough 
data to show it does work. There is an appeal and they are getting 
peojple. 

Tne other part of the answer, I agree with Gordon. You have got 
to have bold stroke. We were talking ateut testing here, and we 
may differ on testing. But that's not at the exclusion of other 
things that have to be done, to look at the research and to see what 
Larry Lazott and Brookover and other people have done with effec- 
tive schools. Effective schools have been effective in urban areas, so 
why don't we replicate what makes an effective 8ch<x>l? You have 
to follow Jim Comer to see what he did in New Haven and now 
what he's doing in Maryland. If it works there, it can work other 
plac^. 

So I think what we've got to do is to see what has worked and 
replicate it, not not say it won't work here if it worked there. The 
other things is to put money into programs that have a good track 
record. 

Chairman Kildee. Yes, Dr. Hertz. 
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Dr. Hertz. AASA has encouraged the idea of money going to mi- 
nority students to be able to go to college. In our own community 
we wrote for some foundation money to do what we call a "celebra- 
tion of teaching", Mr. Hayes, We take our children— and about 13 
percent of our children are minority kids. We realize the tre^ 
mendous shortage in Wisconsin because we just don't have minori- 
ty kids going through our university system. We know what the 
numbers are and they just aren't there. So we have to encourage 
the ^oung people to go in. We have done that through this "cele- 
bration of teaching." We have decided to even move our eflbrts 
from the high school level* where we started them about four years 
ago, and move them back to middle school, because we feel we have 
to get them thinking about it earlier than when they get to high 
school. 

So we started working with them at the middle school level, then 
at high school, and then we tell them, if you go oflf and pursue this 
thing of education in coll^, come back to us. We're selfish about 
the whole thing. But we think it's a motivation as well as a selfish 
instinct, you see. So it can work both ways. We say go off and do a 

good job, and then come back because there are going to be jobs 
ere m our community. So I think they have to see a payoff at the 
end of whatever they're doing, and the payoff is a position. 

Finally, I would just suggest that we are, through our process, 
tiying to show them the economic advantages of teaching. I realize 
what has been said here, that those salaries have not gone up as- 
tronomically. But at the same time there is a good situation eco- 
nomically. There are a lot of fringe benefits and retirement things 
and we try to emphasize thc^ things as well as being realistic 
about the salary. 
Chairman Kjldee. Dr. Kelley? 

Dr. Kelley. About two weeks ago a survey came out on the edu- 
cational and career aspirations of minoritj^ children. It wasn't a na- 
tional but a regional survey. Quite surprisingly, there was a very 
large Dool of people who were interested in going into public educa- 
tion. That was not true five years earlier. There is a pool of inter- 
est there. 

The bill that did not get through the Senate last year actually, I 
think, needs to be looked at again, because nobody is going to go 
into debt like they would to become a lawyer to become a teacher, 

S'ven the salaries that are paid. If you cost out what was in the bill 
St year, you still had to go into too much debt. 
If I were reasonable and kind of knew what my car^r options 
were, I would become a lawyer and not a teacher if I didn't have 
the economic base behind me already. So 1 think you need to be 
aware of what those actual numl^rs are. You're spending money 
but not providing incentives, so you might get nothing out of it. 

On the other end of it, it is pretty clear that one reason you don't 
have more minority candidates for professional petitions all togeth- 
er is because, disproportionately, that 30 percent we're talking 
about come from urban areas and are minorities. TTiat is a problem 
which I think we can identify and have to address, the education of 
all American citizens. 

Again, dealing with that problem is not difficult. We know who 
the people are. We know at a very early age sometimes who the 
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people are likely to be. We know what the problems are, and we 
obviously know what to do about it in man^ cases. We know it is 
not inexpensive in some ca^s. We know it s cc«t effective if they 
become employed, even in terms of the Federal tax exchequer, ft 
has nothing to do with testing. 

If we're talking about balancing, and we're saying all the parts 
of these 15. 19, 21 things we need to do to improve American edu- 
cation need to move in balance, I would like to remind you that, 
compared to other countries, we're out of balance. We test more 
relative to the other things that other countries do already. 

Mr. R.\YES. Thank you very much. 

Chairman Kildee. Dr. Neifl. 

Dr. Neill. In addition to money and school reform, both of which 
are critical parts of this, there is one more. We currently have 
across the country tests that teachers in mc^t States must take in 
order to become teachers. The most commonly used one is the 
NTE, the National Teacher Exam made by ETTS. The pass rates for 
whites are typically in the 80-85 percent range; the pass rates for 
Latinc^ and African-Americans is typically w<»H bciow 50 percent. 
That is, these students do not become teachers tecause they cannot 
pass the test. 

Now, the claim of those tests is not that if you pass it you will be 
a good teacher, but that if you can't pass it, you don't have the 
ability to be a gocd teacher. That is a piedictive validity claim. 
However, ETS has never done a predictive validity study to prove 
the point. 

There is some evidence that exists, some from Montgomery or 
Prince George s County, MD over the last few years of where they 
had to hire teachers who hadnH yet passed the test. The ratings 
they got from their supervisors were as good as those given to 
those who did pass the test, There*s a similar case in the military a 
few years ago; that is, the inability of tests to actually predict what 
we need to know. 

The other thing that*s happening with these tests is that there 
are many schools now, particularly black coileages in the South, 
that have reorganized their teacher programs. They have cut them 
drastically down to a quarter of the size they were before, and they 
reorganized the instruction solely geared to enabling these students 
to pass what is, by many educational experts, like a really bad test, 
the NTE. 

I would say that the NTE is not anything itself that the Federal 
Government can do anything about. However, I would fK)int out 
that i^t food is more regulated by the Federal Government than 
the tests we give our children. I nave nothing against my pot— I 
have a dog and I like her— but I do think we ought to think seri- 
ously about what we can do to improve the qualitv of the testing 
and that there ought to be something in place so that when a test 
has a predictive validity claim to it, you actually know it does the 
job. 

Chairman Kildee. I think we will know this Nation has made a 
really strong commitment to education when in any given commu- 
nity the average teacher is making what the average attorney or 
average doctor is making. You know, you can juoge a person's 
value system, a nation's value system, by how they spend their 



3 11 



297 



money. You go to the average community and the average teacher 
cannot live the life of the average lawyer or average doctor. I think 
that's a very important thing. 

This has been an excellent panel. These last two days have been 
really helpftU to the Congress and Vm sure to the Nation. I thank 
al! of you. You're very, very busy people* people with great respon- 
sibilities, and the Congress and the Nation is grateful to you for 
your testimcny here today 

Ordinarily I keep the record open for two weeks, but because I do 
want to get this record printed up, I will reduce that to one week 
for any additional submii^ons. 

With that, the subcommittee will stand adjourned. 

[Whereupon, at 2:05 p.m., the suHcommittee adjourned,] 

[Additional material submitted for the record follows.) 
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Mr. Chairman and Members of tne Subcommittee; 

The National Education Association which represents 
more than two million education employees in the nation's 
elementary, secondary, vocational, and post secondary 
education institutions. We appreciate this opportunity to 
comment on the issue of a national student test, 

N£A has long opp4>sed a single, standardized national 
test for students. Already, students from kindergarten on 
are subject to a broad array of standardized tests — 
required by local or state poliry — and of teacher- 
developed and administerej tests. Adding another test to 
the mix w^uld do nothing to advance any significant 
education goal, and, in fact, could undermine other 
impoi tant education objectives. 

MEA believes that — In the context of efforts to 
achieve the national education goals set by the president 
and the nation's governors — it is appropriate to examine 
the whole area of assessment and accountability, and we 
agree that students, teachers, and schoo^ should be held to 
high standi^-rds. We simply do not believe that the 
imposition of a national test is consistent with these aims. 

Overview 

The National Education Association recognizes the need 
for ongoing comprehensive evaluation of student progress. 
We believe that student learning oust be assessed with 
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measures directly linked to the lessons teachers teach and 
the materials teachers use. 

Student assessoent should not be equated with testing, 
and cei'tainly assessment should not be equated with 
standardized testing. Appropriately, student assessment 
should involve teacher-developed and administered tests, 
formal -^nd informal observation, and it should include 
assessment techniques that measure a broad range of skills < 
- many of which cannot be ascertained through a pencil and 
paper tests. Student projects, oral reports, and other 
assessment methods must be used to demonstrate mastery of 
subject matter. Hone of these techniques alone should be 
used as the single determinant of a student's future — 
academic or otherwise. 

Repeated studies demonstrate that standardized tests 
have inherent cultural and gender biases. All evaluation 
methods should be rigorously tested to eliminate such 
biases. 

Opposition to national student testing should not be 
equated with opposition to holding students, teachers, and 
schools to high standards NEA supports high standards and 
believe that local, state, and federal officials who share 
responsibility for educational policies must examine those 
standards on an onqoing basis to determine whether they are 
in accord with the current economic and political 
environment and with national and local needs. 



ERIC 




301 



3 



Moreover, NEA recognizes that there is a place for 
standardized tests. But in order to assure that such tests 
are used appropriately^ standardized tests must be 
developed, administered, and interpreted by trained 
personnel. Further, the results of those tests must be 
returned to the teacher and other appropriate staff in a 
timely manner in order to modify the instructional program. 

NEA supports truth^in-testing as an important step for 
bringing about test reform, and we urge adoption of 
procedures that allow test takers to request a posttest 
review of all standardized test questions, scores, and 
rationale for the correct answers. 

National Student Tasting 

The current drive to establish a national student test 
threatens to undermine the two chief strengths of the 
American system of public education: its diversity and its 
ability to give students a second chance. 

Many proponents of a national test argue that a 
national test would merely set goals for a body of 
knowledge, and that local and state officials would have a 
high degree of discretion and autonomy to determine how bet;t 
to achieve those goals, buc the fact is that a single, 
national student test would lead inexorably to a national 
curriculum. If a national test is established — with 
significant consequences for funding, personnel decisions^ 
and curriculum — then virtually every hour of every school 
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day will be devoted to preparing students to do well on that 
test. 

Not only would this tend to undermine the diversity of 
our public school system and the authority state and local 
officials now have, but it runs directly counter to the goal 
of better integrating higher order thinking skills such 
as problem-solving, analysis* and synthesis — into the 
curriculum. Most nations which have a standardized national 
student test tend to use their educational systems to 
discourage diversity. While Americans investigate why it is 
that Japanese students, for example^ tend to perform better 
on standardized tests, Japanese educators are doing research 
in this country to find ways to encourage greater diversity 
and creativity in their educational systems- 

Moreover, nations that use a national test generally 
use those tests as a gatekeeper. A student's performance on 
tests at the secondary and postsecondary level can 
determine, once and for all, their educational and economic 
future. The United States places a high priority on 
individual choice and on giving individuals a second chance- 
In consultation with parents, teachers, and counselors, a 
secondary student in a public school in this country has a 
number nf choices as to whether to pursue a vocational 
track, a basic educational track, or a college preparatory 
track. In Japan and most European nations, those decisions 
are determined by a student's perfonaance on a test. 
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In the saoe way, Americans have a wide array of options 
in postsecondary education opportunities. In the U.S., an 
individual may choose to attend a college or not, may go 
back to school after a short or long hiatus, may pursue 
postsecondary vocational education or an acadeinic education. 
As a percentage of the total population, more Americans 
attend postsecondary education institutions than any other 
nation. We make extraordinary efforts to provide that 
opportunity to individuals because we recc^nize that 
people's interests and level of commitment may change over 
time* A national student test would tend to diminish the 
ability of our society to provide those opportunities. 

Improving studetit Xsssss&ent 

NEA strongly supports efforts to improve student 
assessment as a means of providing useful information about 
the effectiveness of educational practices and policies. 
Meaningful assessment methods should allow educators to 
diagnose student learning needs and select appropriate 
instructional strategies, as well as provide feedback to 
students and parents. They should enable school staffs to 
determine the effectiveness of curriculum and textbooks. 
And they should inform the community about how their schools 
are doing and what the schools' needs are. 

If one can accept that these are the primary purposes 
of student assessment, it should be clear that traditional 
standardized tests have limited use for the changing needs 
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of education. Rather than establishing a national test. 
Congress should support inservice education in evaluation 
techniques, research in effective assessment methods^ and 
other programs that enable school faculties to improve the 
local assessment and, as importantly, to act on the 
conclusions of that assessment. 

Conclusion 

HtA believes that a national student test would only 
confirm vhat we already know — that some students do better 
than others, some teachers do better than others, and some 
schools do better than others. NEA believes that a national 
test would reveal that schools perform best when they have 
sufficient resources to attract and retain quality staffs, 
have a broad academic program, and adequate equipment and 
materials. It would reveal that teachers do best when they 
have solid preparation programs, quality inservice 
education, and support from parents, the school itself, and 
ancillary agencies in the community. It would reveal that 
students do best when they come from families who support 
education and who have the resources to provide adequate 
nutrition, health care, and other human needs. 

Achieving that difficult agenda — assuring equity of 
educational opportunity through adequate resources and 
support from the community and the society at large is 
the only way to achieve the national goals in education 
established by the President and the nation's governors. 
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NEA believes that we have sufficient information to know 
what it will take to achieve those goals. All that remains 
is for us, as a nation, to provide the resources, programs, 
and policies to achieve them. 
Thank you. 
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March 21, 1991 



The Honorable Dale E« Kildc^e^ Chairaan 

Subcommittee on Elementary, Secondary and Vocational Education 
U.S* House of Representatives 
320 Cannon House Office Buildinq 
Hashing con, DC 20515 

Dear Representative Kildee: 

As Secretary of Education for the United states Catholic 
Conference, I would like to take this opportunity to express the 
interest that our department has in the on-going deliberations of 
your subcommittee concerning the iiaple»entation oi the National 
Goals for Education in general, but t&ost specifically with regard 
to the question of a national test /assessment program. 

The issue of standards and their assessment is a complex but 
important issue with many implications. There is a need to 
examine all of these possibilities carefully before taking any 
form of final action. One thing that X believe we want to avoid 
is a piece of legislation, along the lines of a single national 
test as opposed to a more flexible process of assessment. Such a 
single test might be viewed by some as a "quick fix" approach 
that could easily create more questions that it may hope to 
answer . 

While I do not intend to speak for the total private school 
community, I believe that it is fair to say that we are greatly 
interested in these questions and have been trying to assess the 
implications on our students, programs and staffs. 
Representatives of the Council for American Private Education 
have been meeting with Archbishop Francis Schulte of New Orleans, 
who serves on the President's Advisory Committee on the National 
Goals, and have expressed some initial reactions relating to the 
implementation of the goals and specifically our concerns in the 
area of assessment* I believe that in the future^ 
representatives of the diverse elements of the private school 
community may wish the opportunity to address these issuer in 
-detail with your subcommittee. 

The private school community in general, and the catholic school 
community in particular, plays a significant role in thf> 
educational well being of our nation. We have a strong interest 
in any proposal aimed at improving the education of our nation's 
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young peop_e. I believe that wo can offer a valuable contribution 
to any debate on how to attain this admirable goal and therefore 
I believe that the private school community should be included 
wherever possible, in such deliberations and on committees which 
may be formed to address basic issues of interest to the entire 
educational community* 

I hop© you will place my comments In the record of the 
subcommittee hearings. 

Please accept my thanks for your efforts on behalf of our school 
aqe young people. 



Sincerely, 




Sister Lourdes Sheehan, R.S.M 
Secretary of Education 
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National Ai.liance of Business 



March 13* 



Hononable Dale E. Kildee 
Chairman 

Subcommittee on Elementarv, Secondary, 

and Vocational Bdiicatlon 
Committee on Education and Labor 
U.S. Houae of Representatives 
Washington, D.C. 20S15 

Dear Mr. Chairmam 

The National Alliance of Buslnera, in concert with the advisory 
txiard to its Center on Bucellence In Education, has been working on 
the concepts of national student HKcsamenta and testing. 

I would tike to submit the attached written testimony for your 
review and as a formal statement for the record to l>e tnrJuded with 
other testimony taken at your March I4th hearing on the subject of 
national student testing to evaluate educational progress* 

\ look forward to workmg with you and other subcommittee 
Members on thesr issue!u 



Sincerely* 



Wllfiam H. Kolb^ 



President 



Kc^. S<n^ ^ynuH \\\ W^hmyu^i IX ^^ItJS 2HKt I ^X Iti^ lM>i 



er|c^^^ copy available [] I J 
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STATBM BNT POE TlIB EBCORD 
BY 

WiLUAM H, EOtBERG 

ON BBJIALF OP TlIB 
NATlONAt ALUANCE OF BUSINKS8 

RBFORB THB 

SUBCOMMrTTKB ON BLRMENTABV^ SBCONDARY, AND 
VOCATIONAL EDUCATION 
COMMITTEE ON EDUCATION AND LABOR 
U. S- HOUSE OF RKPRKSKNTATIVKS 

"NATIONAL STUDENT TESTING TO EVALUATE EDUCATIONAL 
PROGRESS* 

MARCH 14. t9$i 

The Nationftl Allmfuv* >%( Husifipsv ts nrt ?nf1rp<»nrtont, fjusmoss r.,m 
profit corporation nhosf* missit^n i«t to tn<»rease private sector t r?i in injf 
«n<i job opportunities for fOonomieaUv disadvantaged and long term 
unemployed individuals by tnitidini? snd strengfthening public/private 
pertfiersftips of business. j?ovemment. l&bc^r, education, and communftv- 
!*>ft*ief1 grmips. 
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STATBMRKT FOR THE aBCORD 
BT WILUAM H. SOUBHRC 
ON BBHALP OP THS 
HATtONAL ALUANCR OF BUSIViRSS 
8BPORE THE 

SUBCOMMrrTKE ON BLKMBNTARY, SECONDARY, AND VOCATIONAL EDUCATION 

coMMrrrRBON education anb labor 

HOUro OF RBPRESENTA'nVBS 
•NATIONAL STUDENT TraTlNG TO EVALUATE EDUCATIONAL PROCRESff* 

MARCH 1991 

Mr. Chairman, i appreciate Xhe of^wtunity to submit a statement for the record 
today to provide a business perspective on education reform and, in particular, at>out the 
need to assess our proifress toward edi*eation excellence in thJs country. 

Whtie education remains largely a slate and local responsibility in this country, we 
are now viewing education issues in a national context and as a national problem. This 
concern has led the National Alliance of Business to join in partnership with other 
national business organizations in the Business Coalition for E ducation Reform to work 
over the long haul with education and community leaders to help reverse declines in 
education quality and economic opportunity- CThe Business CosHtion for Education 
Reform includes: American Business Conference. Black Business Council, The Business 
Rcundtabie, Business- Higtier Education Fonimt Chamber of Commerce of the United 
States, Committee for Economic Development, The Conference Board, National Alliance 
of Business, .Siivnal Association of Manufacturers, and the U.S. Hispanic Chamber of 
Commerce). American education quality more widely, and more often, talked 8t>out 
among business leaders today than 1 would have imagined possible jusl five years ago. 

The American education enterprise^ wttich worked well for most of this century, 
has not kept pace with the increasing demands of modem society and our intemationaliy 
competitive economy. 
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By way of Ulustratlofv the futurist Marvin Cetron ht<J ^r,\^ that when the Class of 
the Year 2D0O graduates from high school, the body of knowledge will have doubled four 
times since 1988. And these future graduates will have been exposed to more 
Information duHng the Year 2000 than their grandparents consumed in a life time. 

We are a society wh^e future relies heavily on the <3uaHty of ^r workforce* We 
roust dramalieally Improve l>oth how and whst we teach for all our youth an^ equip them 
with the knowledge and skills for the world they wtU enter, !t Is equally clear, given 
future projections of fewer Isbor market entrants, that we can no longer afford to lose 
30 percent of our youth who at some point drop out of school before h^gh school 
completion. 

Business has experienced first hand, the results of lower American educational 
excellence. U.S. education performance has declined, when compared to other 
IfidufftHalieed natiors. This shows up m the workplace through lower productivity, lower 
wage growth, and a diminished competitive posture in the world market. This explains. 
In part, the compelling interest of business In education, although our interest as citizens 
in the quality of our society goes beyond the bottom line. 

Itie Na tlpaal Coromltment to Education 

All of US just witnessed the dramatic events c/f war in the Persian Gulf, There ftre 
some valuable lessons from that enperience about what this country can do when we 
anticipate and prepare for a potential crisis in a fast changing environment. We spent 
years on the deveiopment of new tecnnology, on experimenting ond refining new tactics, 
new strategies, and, in effect, modernising <?very aspect of ktiowledge, trsining^ and 
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litttitutional stnietures for our national defense to meet new challenges in the changing 
worW - and the results were deraonstrateiL It shows that if we really put our minds and 
resources to something, we can he aicccssfuL We proved with the moon landing in the 

and with the recent Gulf War that when there Is consensus in America, wc get 
things done. Currently, we hove neither national consensust nor standards, nor 
a^essment systems on what students need to know. 

We in lousiness now see our failure to educate our young people to world class 
standards as a major national economic problem which will be solved only if we apply the 
national will, new strategies^ superior technology, amJ new institutional structures which 
can lead us to a comparable victory t>y the year 2000. 

The Industrialized nations who have become our key competitors around the world 
are those who long ago recognized the Importance of education for their economic well- 
being. They each have adopted a national policy end practice for a systematic transition 
from school to work for all ycHith, and, with a high level of pducation skills in iheir 
workerst each has been able to organize work more efficiently with greater productivity 
by cultivating higher skills in front line workers. Their approaches to developing work 
opportunities through high skills and high expectations draws a sharp contrast to 
America's current approach. 

National Standards a mi the A ssessment of Progress 

The i^ational Alliance of Business and other organizations emJorse the notion of 
establishing a framework for national student assessments. I need to emphasize the 
plural assess ments - not one test. We see this as a varted and cumulative system of 
assessments over time. Andt it is a system based on a set of national standards. Our 
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objeettve is for every student to leave school with s dfemonstrated aWlUy to read, wf ite, 
eomptite and perform at world class levels in general school subjects, and also to be able 
to learn, think, and work effectively both In groups and alone. 

The concept of assessment which the Alliance supports would facilitate 
development of assessment systems that provide a variety of ways for student* to 
demonstrate knowledge and skills. We do not want a single national test, b ut, rather, 
commonly held stand ards of Performance and comr€tence» demonstrated throuirh a 
fero^dly understood system of diverse assessment techniques . Students would be assessed 
over a period of years. Cumulative assessment provides multiple oj^rtunities for 
jccess rather than a single high-stakes moment of possible failure. It will enhance 
opportunities for the undereducated and under motivated to achieve high educational 
standards at their own pace, because the criteria for the assessments would not vary, 
regardless of the student's age. Although we advocate national standards of performance 
and a common assessment system, wc also believe that the system should l)e 
sdmlnistpfed locally, not nationally. 

U.S. employers have no reliable gauge of what a new worker knows or doesn't know. 
Likewise, schools do not know what workforce skill demands are. Kortunstely* we are 
finally seeing national and local efforts to bridge the knowledge gap between curricula 
learning and the workplace. 

The National Education Goals Panel is dpvr loping peoommendHtion** on standards 
and assessment systems. The work of the Secretary (of tabor's) CommtJision on 
Atnieving Necessary Skills (SCANS) will provide an important starting point by 
identifying the types and levels of skills needed for success in ihe competitive 



314 



N«lUMttJ AUunc««fB<i«in»tt Page S 

workforce. The National Center on Ed»H?atlon and the Economy is developing a national 
assessment system which we think has real potential* 

Once new assessment instruments are developed, based upon new goals and 
standards, they will take time to t>e widely disseminated, used effectively, and accepted, 
Ev<fn when implemei.ced. substantial chants in student performance will not be quickly 
forthcoming. In fact, some of the new efforts may fail. Restructuringt by its very 
nature, rtsts on a willingness to experiment. Like businesses that restructure, efforts 
wUI not always succeed* but can point to more effective future practices. 

Business leaders anil educators are already combining efforts to define the 
standards of competence for entry into the workforce, Connecticut, New York. 
Caiifornia, and Vermont, are developing new assessment tools and systems, ones that are 
so comprehensive m the information they capture that they can enhance student learning 
and can even form the basis for teacher and administrator assessment systems. They 
i^tsnfv learning goaK, and more thoroughly measure the higher level skills that business 
and society require. 

An assessment system should be viewed as something developmental that you learn 
from and refine aa you do it. in fact, such systems should continual! ^ evolve, because 
the Knowledge needed to succeed in life and work will change^ 

Changing the accountability system in education through assessment will also 
affect the management amJ administration of education, curriculum and instruction and 
educational professionalism. The development of assessments cannot be separate from 
the professional development of educators, because it wiU have an impact on the nature 
of instruction and curricula content. 
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National Leadership and the Feder^ Rote in gitocstloo ft^f orm 

We still lack many of the tools necessary to achieve broad-based and iastir^ 
success in our efforts to reform education. The establishment of national goals by the 
President and governors was a laudable first step toward defining national expectations 
for the education enterprise. These help to articulate where we want to go. Now we 
have to find the ways to build the national will to achieve these goals, and we need to be 
able to measure our progress along the way. Reaching the national goa's must be a key 
nationai priority. I recognise the complexities involved, but we must articulate and 
apply a clear national strategy* have more cohesive leadership, and provide the necessary 
resources to succeed. 

Currently, we have neither a national consensus nor standards, on what students 
need to know. We are told by teachers that they do not have a clear idea of what they 
should bp tea<-hing differently. We are told by parents that they do not know how their 
children are doing compared to other students, let alone among other industrialized 
societies. (In fact, parents are so confused that 75% of them believe there is a major 
education crisis* but 75% of them also think that the school their child at* ends is very 
good.) And students understand very little about what they will need to know to succeed 
in contemporary society. 

First, national standards need to be t^stablished that set j^ideposts. a framework, 
and the expectations for what students should know. Standards should define what they 
need to know by age - for b^mg a citizen, for going to work, receiving additional skill 
training, or attending college. 
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We should have the abilily to sccumuiale information about where Individual 
students are in meetli^ these broad standards. That information should be gathered by 
school, t>y school district, by state, and nationally. Parents should be fully informed 
about how their children are doing. 

The education standards we ^opt should, of course, be competitive with other 
nations, but be designed to meet U-S. needs first. Our workers must l» among the finest, 
if we expect our businesses to be as efficient as our international competitors, many of 
whom have national educational assessments. 

The U.S., like Great Britain and other nations, need to address this problem 
directly. Britain has established a national curriculum that ail students follow. It is 
standardized. Teachers have national crilcna and assessments to measure student 
progress, Britain went at the problem directly by specifying curricula at the beginning, 
rather than having educational standards and assessments define the changes to curricula 
at a state and local level. What the British have done would be a revolutionary concept 
in this country, i use Great Britain to Ulustrate how strongly other nations arc 
committed to the solution of their education problems- 

We should be wiOlng to ctiallenge a few of the traditions in education by having a 
greater national api^oach, while .till leaving most policy decisions to state and local 
levels. We^ve matured polltieaUy. We can talk now about national curriculum in this 
country, based on standards we establish. The important distinction is that this would 
not be a federal curriculum. I believe there is a role for federally developed criteria, 
standards, or syUabi against which state and local curricula can be developed. 
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It is my worry that we are not serious enough yet about making critical changes in 
educatiOHt and not committed yet to achieving our national education goals. For 
example, the four h goal is to be first in the world in math and science education by the 
year 2000. For us to get there, it will require a national commitment comparable to the 
challenge of putting a man on the moon in the 1960s. I sense that we are still unwilling, 
as a nation* to think about the kinds of changes that would be necessary in policy, in 
structure, and in funding to achieve these goals. Without clear national and state 
leadership* educators are not likely to get behind the efforts with any sense of national 
mission. 

The federal government^ for its part, must be much more pro- active in Its 
leadership to set tl>e tone, set the agenda, and motivate action. There are several 
federal roles which can be played by the Administration and the Congress. The federal 
government should be undertaking a '*moon landing^ leadership role in developing 
standards for meeting our educational goals, so that there can be no doubt at)out what 
states need to do and where children rank in terms of what we have to achieve by the 
end of this century. 

1 firmly believe that there is a federal role in this effort as the lead partner in 
developing information, establishing standard and devising assessments. 

We are now at a juncture in our efforts to reform education where we recognize 
that we must act as a nation on a national challenge, and yet the needed actions must 
primarily be taken by 50 States and 15,000 local school districts, and, ultimately, in 
83,000 school buUdings* The federal government has historically been a relatively 
S^ssive force in education and provides less than 10% of the fundf for education. The 
challenge is to define a pro-active national le adership role for the federal government 
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without displacing or supplanting the trsditionai State and local responsibilities for 
education. 

In addition to our strongly held belief that the federal government must play a 
cHtleal roie m the development and establishment of educational standards and help to 
design a system of assessments, there are other specific activities appropriate to federal 
responsibilities In meeting our national education goals. 

SotionQl Leadersh ip throu^ Rege<ych and Inforrggt ipn Oeyelop menU The federal 
government nas a traditional, and accepted, role in supporting research upon which new 
knowledge and innovation in education are based. This roie needs to be expanded in the 
90's, as we search for new methods, develop standards and new assessment mechanisms, 
and to otherwise work toward meeting our ambitious goals. What is needed is a strategic 
approach to information development and the use of data in relation to national goHls. 
We should carefully. t>ut pro-sctiveiy, develop the national capacity to provide the 
knowledge and basic data from which Ihe ennre educalK;n<'ji svstrm i'nn ilr.'iw. 

The single largest pot of research money that could !>e directed to these efforts is 
under the Education Department's Office of Educational Research and improvement 
<OgRi|. What we should buy is a new strategy for the future education of our youth. The 
department should be an important catalvst for change, and should underwrite much of 
the developmental work that is nece^ibarv. 

My sense is that the research agenda of this office has been to fund a bruad range 
of small, discrete research projects which do not add up to a strategic plan or vision, 
does not provide information which the States and local schools need for their efforts to 
restructure education, and is not widely or systefiiatically disseminated when completed. 
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\M»onftlAi]»»rK«t}fBu»tn«» 10 

We are missing an app(»i unity for the federal government to provide the leader^lp in 
reseafch on education restruettirtng amS national goals* 

Early Childhood Deveioffnen^^ This ii» a programmatSe area in which federal 
leadership has had tremendous impact. It Is the best example of HUing a gap in the 
traditional systems of EDandatory education. Barly ehUdhocd development is an 
important new concept in education, as a strategy of prevention, of which the Head Start 
progpam Is a part. TTw Committee for Economic Development {CEO) has provided 
leadership on this issue, and has not only educated the tetsiness community about the 
Importance of preschool education ami health care* but also has argt»ed convincingly for 
purwinf a strategy of prevention In public policy. On March e» 1991, five Chief 
Executive Officers of major VS, corporations testified before the House Budget 
Committee on full-funding of the Women, Infants, and Children (WIC) pn^ram. 

We re<M)gni2e how criticei early childhood education can be. It has a direct impact 
on social skills, educational achievement, and self esteem. We at the Aliianee see 
Investments in early chiUSiood programs, like Head Start, as an important weapon in the 
fight agaimt the pfol>}ems of school dropouts, drug atmse, crime, and teenage pregnancy 
and for that reason we are on record in support of full funding for Head Start. It has the 
potential, over the long term, of aUowIng ta to redirect limited federal dollars that 
otherwise might have to l» spent on "second chance" systems to repair the damage that 
could have been prevented. I woutd also argue that we are at a point where the costs 
could be shared with the States. We know that about 30 stat«^ have enacted various 
types of early childhood programs, 9 of which are desigml to supplement Head Start. 

Now, with the governors recognising a national goal related to early childhood 
education, and with the states havii^ ^mary responsibility for public education, perhaps 
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we coukS move to a greater level of shared respoRslbility m early childhood developmenl. 
There is precedent in virtuaiiy every other pn^rsm of federal assistance to edueatiom 
The ciosest comparison is the federal Chapter 1 program, which predominately covers 
poor children in the early years of elementary sehootr in which costs are shared with the 
states* 

Build Linkages in the Broo-d ]imQe of Federal ficfcicotton Prooroms . We must take 
much more care to rationaliee how individual pr^rams are linked in a cohesive 
continuum of education development. Individual federal education programs must be 
thought of in relation to each other. For example^ Head Start cannot be separated from 
services under Chapter 1 of the Elementary and Secondary Education Act or we Hsk 
losing the gains of one program during ttie application of another. Each program should 
build logically on the progress of the others. We stiU need to empnasixe educational 
services to disadvantaged groups* but the policy must be to build on linking education 
programs. This can have an effect in reducing the bureaucracies which have been 
estabiished over time at the federal, state, and local !evets. 

integr§ie He a lth and Human Service Proarc ^ with E cju^ation. Educators are 
often the first to lientify health or other ^>roblem5 that are preventing children from 
learning. But, they are often helpless in addressing these problems. Federal and state 
iegislation must put a premium on assuring that health and human service programs are 
readily available to children. This can be accomplished by insisting that sij<»h programs 
demonstrate how well they are tied into the schools and responding to the problems 
identified by school teachers, counselors, and administrators. 

g ytQbf fsft 0 getter School go iVor^ Trar^sition . For several years now, various policy 
studies have focused on the failure of our society to provide school to work assistance to 



321 



Pftge It 



the maioritv of stmleflts who do not go on to college. We are not naive about how 
complieated this issue is, but it deserves urgent attention bf tlito Subcommittee. I 
understand that several members of the full Committee have already expressed Interest 
in working on such a i^oposaL In my view» it involves not only integrating opportunities 
for woHc experience with school to give relevance to classroom learning ami to motivate 
students Cas the Eur^ans do), but also to assist students with the skills for finding 
meaningful employment. America preperes only a tiny fraction of its non-coUege bound 
students for work. Other industrial nations have mult i -year career educational programs 
that prepare stiKlents to operate at a professional level In the workplace. The enactment 
of the "Tech- Prep** or ^Two-plus-Two" program in the vocational education 
reauthorization last year is an important step in this direction and may serve as a model 
for a more extensive system of occupational certification. The Alliance intends lo 
develop more detailed ideas on the federal role in a school to work transition^ and we 
will work closely with the Subcommittee during the this session as our work progresses. 

Provide a Safety Set for Those Who Wnulti Ot h^wtse FaiL This is an important 
and traditional rr.e cf the federal government in education to a^^ire equal opportunity 
and equity of services in education* We would like to see this role broadened In the way 
help is provided to school dropouts. There are a variety of existing programs in this area 
that need to be linked more carefully Into a cohesive strategy. Ail students should be 
guaranteed the educational attention necessary to gain mastery of a standard set of 
educational skills 5y age 16, or as soon 8S possible thereBfler. 

Insist on AcctH mt<^ilily * One critical lever that the federal government has over 
its investment in education is to carefully structure and insist on accountability. This 
means not only fiscal accountaoility, but also accounlabtJtty for solid results. The 
Chapter 1 accountabiHty standards are an example of what is needed. We are not 
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prepared today to recommend specific methods to achieve occountabiUly, l>ut w<» do 
beii^ve that rewards and consequences should t>€ a part of education program legisletion. 

The Btfgtness R ole 

As for the business rolc» the business community muat be Included os ft partner \n 
this endeavor. Busmess can provide information on competency levels for jobs, so that 
learning experiences and curricula meet the needs of a dynamic society. Business can 
provide technical assistance in developing standards, assessment tools, and strategics. 
Business can work closely to help in training of teachers to apply these new assessment 
techniques. Business must also be nn advocate for a comprehensive system of 
assessments and continue to press for systemic change* 

Business leaders can be instrumental in keeping education high on I he public agenda 
in their states and communities. They can be strong Bdvocates for the transformation of 
rh#» si'hot-ii^. Thf^v «»an help raise the stt?tits af <*duc«tors who, fettling pow<»ri<'ss find 
frustrated^ often lose any incentive to press forward. Joint efforts are necessary to 
address the spectrum of education issues in a coordinated and focused approach. 
Business leaders must work coUaborativety and over the long term with educators as well 
as community leaders toward common goals. 

In Cpnetusion 

Mr. Chairman, this is a complex agenda for change. Despite what seem like 
insur mount ab!e obstacles, a growing current of public opinion demands change in 
educaiioti. AU Americans must play a part. We in business are preparing ourselves to 
play an important role in achieving significant change and improvement. 
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Our long term agendSt thiwigti the Alliance's Center for Excellence in Edueatfon 
and with our piartnera In the SusSness Coalition f&p Education Reform, is to HraS and 
impiemvnt more effective ways for business Involvement. 

This education reform effort requires strong national leader^ip from the federal 
government in setting the vision and the goalSt fostering change, ami u ensuring that all 
the stakeholders carry out their appropriate roles. 

As America struggles to Improve its education system, so that our society can be a 
strong and informed democracy and can compete in a global market place* it is essential 
that we are able to assess* in a meaningful way, the educational progress of our students. 
We are united In our determination to prepare our children to meet the challenges of the 
future^ An accurate* and fair a2»essment system Is an im;»rtant tool toward that goad. 
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March 21, 1991 

The Honorable DaJo E, Kildoc 
Cha insan 

Subco&mittco on Eloraentary, SecondAry 

and Vocational Education 
Cofiu»itte& on Education and Labor 
320 Cannon House Offico Building 
K^shinqtun, DC 20^15 

Re: Conisentft of the Asso<,iation of American 
Dear Mr. Chasmwin; 

During the March 14, 1991 hearimj on "National Test inq; ProK and 
Cons,** the Subcoaioittee aqre<»d to hoJd of>en the reccro until today 
in Older to permit other interestGd partieti to subait the it View« 
on the Issues, The Ajssociation of American Pi-blishers ("AAP-J 
approciatts the opportunity to submit its viows on national tos;tin<? 
to this Subcommittee through thia letter and attachroontt;, 

AA?» is the principal national trade organization representing B»ore 
than 2 3S coramercial publishers of hardcover and paper bjck books; 
professional, technical and scientific journals; computfr sottware; 
and educational and classroom materials, including tests and 
evaluation scoring materials. As such, niewtoer« of AAT historic-ally 
have been involved in meeting the needs ot school districtr, and 
«tate education prograSQS for various ty^ws of a<;nef;ninont 
instruments that have been utit?d to ast;i:>t in tho iBeai.ui ^rn^nt and 
evaluation of the nation's school children- 

For the future, AAP stands ready to play .^rtJve rote in thf 
debate over Tiational testing issues, so that coir.not i i 1 test 
publishers ©ay contribute to a cony^ensur, solution for impiovmg the 
ntition*8 educational system. We recognize our responsibility to 
bo as creative as necessary to help our society moot thoije new 
torroidabie and exciting challenges. In that rt>spc'Ct , AAl'*;. lefitin'i 
Contaittee has already uiet with Governor Hoiaer and intends to 
continue those productive iiscuRsions. We also will open <i 
dialogue with members of the business community to disrus?^ how 
commercial publishers may work with business iead€»rs to better 
understand and /-espond to the perceived educational problemr* 
associated vith the coropleK issues surrounding assessment. 
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The Honorable Date C fCilJoe 
March 21, 1991 
Page TVo 

»0 aiKfltJI KXTIOHXl. TEBT 

AAP fitrongly op|>os«s ary ginyle test to be used nationwido. 
However, AAP fiui^rts tho concept of a nationwide assessm€?nt 
program* provldod it Doets specific crit<)ria that, fron the 
experiences of costBorcial publishers^ are re<|uired to ensur<T that 
all univer^lly-Ad&lnistered assessaents (ao distinct from 
aeeesomonts of student sAs^les which jsay bo intended for us^ 
priaarily to isonitor school systejas and/or trends) will be usc^d to 
improve student instructicn and school curriculua. In this way, 
AAP also believ<3S that sore accountability can be brousjht to the 
aseessoent system* At the ease time, AAP firmly believes that test 
diversity and conpotition ahoulu continue to be integral to any 
refora in order to assure that the best quality products are 
available. Publishers want to assure this Subc^omsittoe that the 
use of multiple tests can still provide uniform comparative 
inforaation. 



miffCTlf MfigggHgKI CRXTBRIA 

1. laifixfi Rust be cittltAfiiiL^tEalina instruc^jnt-'Si. .^^v^iiii^iii?- 

Different tests s^-rve different purposes. Some tests are designed 
for diagnostic purposes, others for accountability reaf^ona and 
still others may focus o:> i^onitorin^ progress. No or > test, indeed 
no one type of test, can accomplish all of the div*?rse goals and 
objectives of this country's educational systeffl. Nev«»rthe}osti, as 
indicatcKi above, multiple t08t*j used in a coordinated fasshion can 
still provide uniform coaparative data. 

A test should not be u&ed for purposes for which it w«is not 
designed, nor should a test becoise the sole basis lor dotornining 
how a student is doing. Consequently, AAP urges that the first 
important criterion for a national ass«ss»ont prograju reust b«» 
adherence to the concept that a sirujle national test not be used 
ior purpof:es of detenaining progress toward .ichieving n*»tiona) 
education goals and standards. Additionally, varieties oc 
assessment formats, frois normod* referenced, standardized tests to 
performance^based assessments, have a role in the mix. An 
explanation of tho various types of tests and formats is included 
in the attached testimony of Burt Faldet, given before the Stfnate 
Subcosmsittee on Education, Arts and KuRt^nitxes ol the l^bor and 
Kuman Resources Coms^ittee on Harch 7, 1991. 

2 . Each asse s nment mus t .ii£^«temcii^ t £aXfi4 _tP- l??.y a iX^. r v J i<t b i C . « 
bji , g^{^^f rep . Scrutiny, criticism and debate about tetits and the role 
they play in tho lives of studencs and whether they are biased or 
exclusionary is not ^ new or novel deveiopment of the day. Those 
very issues have confronted the testing community, including 
developers, users, administrators and others, for years. As a 
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The Honorable Dale E. Kiliicie 
March 2I« 1991 
Pa^e Three 

result, voluntary stgiKl ^rtiB ^or Educational and Psvchoioaical 

Tefltinq ver« adopted by the ABerioan Psychological Association 
(along vith tiM AMrlcan Educational Reeearch Association and the 
National Council on Noasuresent in Educaticm). The g^andarda set 
forth the professional ly-acoepted instructs and procoduree to be 
fol loved in dsvoloping psychological tests ^ doterainlng their 
reliability and validity^. iaplmBsntinq their use amS describinq 
research results. 

APA * s standards have been w ids ly cited with appr ova 1 by 
Congress « state legislatures, courts and researchers. The cost 
recant endorssBsnt occurr^ during the House consideration of H.R. 
12 BS, the Higher Education Technical AMndaenta of 1991, between 
Chairsan Ford and Congrsssaan George ffiller {page H. 1612, Karch 
19, 199X) . Under the6»e StaiKlards . develc^rs and piJ^li^ers of any 
assessment inatrusent, regardless of whether it is sultiple*choice 
or perfomance-based. Bust present research on scientific validity 
(whether the instruMnt predicts Wiat it is designed to predict), 
reliability (Whether its results are repeatsble) and fairness to 
protected groups (vhethsr there is bias against any identifiable 
group of test takers} . 

3. Any national testing progrfflB suat encour age open competition 

among all test devaloper a- including cogsorcial BttfejiPhgro* 

Rcrgardless of the types of assessaents to be Included in a national 
prograa, couMrclal test publishers have an extraordinary 
bacIcgroQnd and expertise in producii^ instrusents that cosport vith 
the APA's standards and with the equally applicable Cflde Fftir 
Tostino pyacy^ices in gducatimi. The CsUfi is priaarlly focused upon 
profess ionally-devel^>ed tests used in and by educational 
institutions. It is the conitaent to extensive research and 
developaent by c osser cial test publishers that aaKes them second 
to none. Because these are private sector cospanies. thoy invest 
significant sonies each year in conducting scientific research. 

Similarly, this Subcomittee should be aware that AAP sembers 
have been involved in offering perforaanced^based assesssents 
(e.g., writing and iisteniAg ssssssssnts) for years. Soma 
publishers are also offering portfolio assassaents as 
supplaaents to currant test betteri«s, Aooordingly, cosaercial 
pubiiM>er» have already dsarostrated the initiative and 
rasponsivsness to ths daaands for parfor«sncsd-bss«d sssessaents. 
Any national prograa should cask to continue an open coapetition 
in this devalopasnt and adainistration of assessaent instruaents to 
ensure that the best products are available to seet eserging 
educational nssds. 

4. A national aseasaaent svstaa aost imimleaent. not gupplant. 
eitistino s^t^a and local assaasa^, AAP faels strongly that 
ccmtrol ovar the attsinsent of national goals and ssloction of 
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assesamento Dust remain vith the state and local authorities. 
Theae ontiticjs ara in th© fc>ost pof^itlon to make the nectissary 
refinef&ento to any national asoofiSBent program that vill onsure 
that it is tailored to th« n«edB of the local conmunity which is 
being served. Aa several witnofiseii have previously tostifiod 
before thie Subcotaaittee, thle flexibility is important to 
preserving the best in the American educational tradition. A 
national assessment proqran should not replace state and local 
testimy prograBB, which may serve other purposes, particularly 
diagnostic. 

^' State and local entities must intoorate tho n ational asf^css^roont 
fivstot^ with on-goin g at^aossroent - Although local control must 
reside with the state and local education agencies « they should 
have in place an explicit and comprehensive program to integrate 
and coordinate a national assessment system with on-going 
assessTBsnts , AAP submits that such an integration approach will 
provide a mechaniss to ensure that all universally-'adroinistered 
assessments will be linked to the is^rovcment of instruction. 

6 - lUa , „ aas essroents n^unt be administered and t he. results 
iaterpretcd in accordancg with the apa's sta n dards and t he C ode . 
Again, reliance on the adopted standards already in place must be 
assured, so that all aspects of tho national assessment program 
can function together. Accordingly, it is important that test 
administration and test interpretation egually xncet thp hiqh 
standards currently set forth by the APA's Standards and the Ceslfi. 
It makes no sense for the development of various assessments to be 
prodicated on these standards and not have that action followed up 
by requiring those who administer the tests and who evaluate the 
test rdsults to conform as well. 

• Much more emphasis must be_iii^e n -to as sc ssme nt t r a i n i no . 
Further to tho notion that test administration and interpretation 
must bo as rigorous as the development of the national asG«>fisment 
program, aap recommends that state and local education agencies 
should have in place a program to train teachers and administrators 
on the use and interpretation of assessments, including but not 
limited to the national assessment system, Research 'i^udies have 
shown that teachers do not make full use of test rerults, simply 
because they have not received adetyuate traininq on these 
principles. Out of this point also derives the added benefit of 
improved accountability, since better informed teachers and school 
administrators will bo better able to report the results of 
assessments to pupils and parents. In fact, the general public may 
also benefit from the greater awareness and und*?r«t and inq of the 
role of testing in our society. 
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Bafora any national a»#«M»«it ayatOB ia is?>laMnted, ho^-var. tha 

S^tM. sSSco-lttaa to pay partlculw attantion to tha 
to t»^a «in»aB«ia on aavaral kay itaas tlwt will »>• crucial to 
Sa^Saia oftoTnitienal proqra.. Such a oonaanmis wat involva 

inttimt li^udlnQ axparta In aasaaaMit and ■aaauraMnt. For ita 
Mrt MP Is^rS^Sy daalroua of eontrlbutlng «anpo»ar and 
?!S^;»«^to au^ M affort and - -ould valc~a th. opportunity 
tTdlaSaa vaya for aceoapllatolng thia av««»a for 
wnaai^awlth thia subcoaaittao or any of tha qrotipa that appeared 
•t tha bearing. 

The following iaauaa are thoaa upon which ag - ^t should be 

davelop»ant of a national aasaaaaant progra» that can ""^ J'J" 
aSjievTthe qoala and objectives advocated by the hearing 
witnaaaast 

1. The ssatent. objectives, and sequence to be 

taught and asaeaaa^^SrWell ae on tha interpretation of tost 
rasSlts. ..g.7^ttai;««,t or »chiave»snt levels, pass/fail levels, 
collega bound vs. technical, etc. 

2 n>a purposes of a national assassBent program, e.g. , 
syste. ac^untabili'^r^ltorlng progress in tSi 

aducation qoaI«; •vcluatin^ th« results or x.n9 
^f^ui«X7f iMtr^lon, -valuatlng^and iaproving curr culu., 
dia^lnTlndivldual atrangth. and «*kneaaas. or certifying the 
tiCSTTof^cMi^nt and <Amty upon ending a««.ndary '^^c^tion 
c!I»,.lSl«rv to irtwt purposofl are served, AAP feels cojapelled to 
r^ ^ ^iTl.^^^ ataxes- a^untabllity testi,^ »ay 
provlda graSt leverage <or changing tJia educational systeo, but it 
Buet be approached cautiously «o tbat thor« are not distortions or 
™inte^ ^TS^«nc« andV. that it vill produce i»prove>»cntB 
In instruction and learning. 

3. The voluntary or gompuleorv nature of participation 
m a national asaoasBent progran. 

4, Who pays for the development, administration, 
scoring, and ether costs of the prcqran. 

5 The asJOlS of assessBont, e.g., testing all students 
or a saBple'f testing annually or at intervalsf testing all gradoa 
or carton gradas; testing all students or oKcluding or P'^vld ng 
separata tests or score, for soae (e.g.. non-public schools. 
Chapter It learning-disabled; ESL; etc.)- 
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6. The iMq^cngY of adBinlBtration and of revisions of 
the assoaaoent to assure co®p«ral>illty of results over tiiD€», while 
encouraging a dynaaic and diverse aeAesesent program. 

i « 4 11^; The aacijiltx or aCggflalhiUt^ of the assoseaento, 
i.e., will test i ten* be secure or rsleafiked to the public? will 
test ite»s be protected by copyright? win tost items be released 
to be issbedded in other asaeosMnt instruaents. 

8. The ggvflting or scaling of different assessiaents, 
particularly assessments using different formats (e.g., muUipie* 
« porformanco assesswents) , in accordance with the 
fftamlttraa ' on this point, AAP notes the difficulty of the 

D!l ^.^^ of equating or calibrating different assessment 

instru»onts to an "anchor* test is not to be underestimated, 

9. The choice of Xaxjaat of the assessments, fjoultiple- 
choice, perfonaance, portfolio, or combination), is a function of 
a variety of factors, including cost, purpose of the tost, timing, 
otc. Each format has advantages and each has disadvantages. it 
is not accurate to state that a performance or "authentic** 
assessment is the -best" assessiaent format without qualification. 

On this last point, AAP is in extreme disagreement with those 
Who advocate the need to completely remove th© use of standardised 
noTOod- referenced tests. That ir akin to "throwing out the baby 
«^ ^i!f water." The use of multiple-choice tests has 

Sy^r?!fn.^'«^^''K i^^^J^tion on an PblPgUVP, valid, non^ 
discriminatory basis. This Subcoauaittoe should not, any more than 
some educ*?tora and researchers, "shoot the messenger- when the 
message may not be to their liking, 

PEYELOPHgHT OF WATIO>aL STJfflfi^fipp 

Finally, AAP assumes that this subcommittee recognizt»*» that the 
task of developing the instruments for use in a national assensmcnt 
program is secondary to the task of establishing national education 
scandards. Here again, it is the challenge for every ^^roup 
involved jn the educational system to contribute to a consensu'.- 
building effort as to what every child should know in any 
cl"^^''''.^./*''^* ''''^ particular grade level. AAP hopes 
that this Subcommittee will assure that this effort does not become 
the classic case of "putting the cart before the horee." It is 
premature to develop an assessment system until a consensus js 
reached about what is being assessed. 

AAP believes that the participation of commercial pubiishers in 
this important task can be utilised at critical points jn tho 
fonnulation, development and delivery of appropriate asr.e-;sment 
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d»vftlop«d and avftilabl*. Bv«in thou^ WU» •ffort profilM9 to be 
lanqthyT Perhaps •vmn on tb» ordwr of tl» tw-y»ar Pfojraa 
d«v«lowim outlines by Dr. Lmurw* Rwiick, eewrcial publi«hor» 
m c€iSltt«d to insuring that th« national aaSMnant progrMi will 
bm adBlnlatarad, and tba raaolta intarpratad and uaad, in a 
atandardisadf unbiaMd way- 

For all tha abova raaaona. AAP raapaotfylly auggcata that tha 
subconittaa'a attantion abould focwa firat on tha naad ror a 
eonaanaua of aducation atandardw and than, aacondarily. on the naad 
for a national aaaasaaont progras, rathar than on a singla national 
taat. MP aaabara, who currant ly iso^iaa tha largaat aagsant or 
tha aaaaasMut del ivory ayataa in placa in thia country, ara 
willing and abla to r^pond to tl^ aaarging «^ ^.^^f^*!^-; 
viav of what fiultipla aaaaaa»ant inatruaanta, and poaaibiy thair 
dalivary tachnlguea, will ba da»andod by tha yaar 2000, Juat aa 
thay ha^a dona ovar tha laat gaartar of a ^^^•J'^" 
*nav* aaaaaaaanta and vogua education raforaa have required aimilar 
coaaitvanta of reaonroea. 

If further hearii^ are acheduled, we would walcoaa the chance to 
report to thie SubcoBBiittee on the prograsa of thaae of forte, in 
addition, we would be plaaaed to anawer any questione thie 
Subcon&ittea aay have baaed on thaae oOTDents. 

Reepeotfully aubaitted. 

ASSOCIATTOH OF iUCERICAN PUBUSHERS 



Hicholaa A. Veliotea 
PTMident 



AttachBxants 
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AMOCiation of AamtCBn Pubtl«h«n, inc. 



1 Connecticut A¥«nv8 NW §700 
FAX MS^M 



STATEME N T BY BURTON W FALDET 
OK BCTALFLOF_TH£ A SSOCIATION OF A nERiCA.4 PUBLI SHERS 
BEFORE THE SUBC OWITTEE ON EDUCATIOK, AR TS AND HUMANITIES 
COMMITTE E ON LAHO R AND HU MA N RESOURCES 
MARCH 7, 19 91 



Mr. Chairman and members of the Committee* by name is Burt 
Faldet. I appreciate this opportunity to appear before you 
today on behalf of the Association of American Publishers. The 
Association of American Publishers C'AAP") is the principal 
trade organization representing more than 235 member firms that 
publish hardcover and paperback books; professional, technical, 
and scientific journals; computer software; and classroom and 
educational materials* including tests and evaluation and 
scoring materials. 

I am President of Test Consultants. Ltd, which provides 
evaluation, design, and implementation strategies to education 
and business. Our clients have included commercial test 
publishers* the American Institutes for Research, IBM, as veil 
as individual school districts. From 1965 to 1987. I was with 
Science Research Associates, a commercial test publisher, where 
I was involved in a variety of positions in the development, 
publication, and use of standardized tests for schools and 
industry. I also have taught undergraduate courses in 
Measurement and Evaluation and secondary school science, and 
served as a School Psychologist and Director of Pupil Personnel 
Services. 

There are several points that I would like to discuss 
today, from the perspective of the publishers of such tests, 
about the development and use of standardized tests in 
elementary and seconda ry s chools — whether for the purpose of 
evaluating progress toward national education goals or tc an 
individual pupil's deve 1 o jNt>ent . My statement does not addrests 
higher education, employment, or military testing. 

The first, and what I hope will be the most important 
message I leave with you today* is that developers and 
publishers of standardized tests should be seen as part of the 
solution for improving the quality of educational instruction, 
not as part of the problem. 
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The second message is that test diversity and competition 
should be encouraged to assure improved education and improvea 
assessment instruments. Different objectives are served by 
different kinds of tests — no one test can accomplish all ot 
the diverse objectives of our diverse educational system, it is 
a serious mistake to try to make tests do vhat they are not 
designed to accomplish, or to use tests as the sole means for 
assessment in most situations. 

Finally* I vant to assure the Ccawnittee that test 
publishers — working with the educational consnunity — are 
expanding and improving their testing products to meet 
ccmtinually emerging educational demands. Test publishers are 
ready and able to develop and administer whatever assessment 
instruments are desired, including assessments to measure 
progress toward attainment of national education goals. 

WHY TEST? 

Keasurement can be relatively exact — but a number has no 
moaning until someone makes a judgment about it. That la the 
difference between measurenent and evaluation. There are many 
ways to determine health; a number on a ther^meter is one 
indicator, bu^, it takes someone to exercise judgment as to the 
significance uf the temperature shown, and to take the 
awropriate action as indicated by the reading on the 
thermwneter. It would be . -prudent, however, to rely entirely 
on temperature to make a (diagnosis of the patient. 

Why educational testing? Testing is of value to the 
student. It serves to provide swe information that can be used 
by educators and parents to identify and respond to the 
instructional needs of individual pupils and to improve 
instruction of individual pupils. Testing is a means to assess 
progress toward specific educational objectives, as evidenced by 
what pupils can do in terms of skills exhibitai. 

Testing also serves broader* institutional goals. It 
assists in assessment of long-range effects ,^^^|®L,iS^^^^^^^ 
educational program, enabling coiqparison of (1) performance over 
SSI and ?o lhaSes in the instructional program or to changes 
in population characteristics and (2) performance across 
different subject areas, such as mathanatics and reading, to 

determine strengths and weaknesses » needs for program 

modificatiw! ol changes of emphasis. Testing is one means to 
evaluate performance for accountability purposes. 
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is Jina^in^f K Whether children are learning what 

►LEfiS? '*"9ht have changed over the years, juat as manv 
techniques and objectives of teaching have changed 
example, standardised achie-zement telts and nu^roGs o?her tvMs 
c asI^bJ^Uss^bfllr"^ teacher-made test'^^fnisfeJlS o^^' 
lecvina ^ ™?l!in; P^^fof"'^"^® portfciic tests are 
serving to complfflnent and supplement multiyle-choice formats. 

LIMITS TO TESTIMC 

r^a-iL""®^^ eophasiied. however, that there are limits to 
testing. When testing is used in "high-stakes" situations and 
^I^ri^Lr* * ^^"'P^® -pass/'.ail" barrier to stuSenCB or 

to reward or punish teachers and administrators, when the 
pressure becomes so intense that there is "teaching ?he test" 
^^^^MS"* teaching the skills .,nd concepts that are beinf 
evaluated, when test scores become the sole criteria for 

performaP;.e or potential or the effectiveness 

Tests are a necessary but not sufficient means to assess 
^5°'^'' ^" e""'' abilities. 5ha? 
o,!r™« "f*"'*^ ^' inclusive of all of the desired 

outcomes of instruction. mwb**.™ 

are ™L«2X be used as a partial basis for evaluation. Tests 
lit ""^^^ certain basic skills and abilities and 

tlL^l intended to measure to'-al achievement in any given 
«^™e I«r'I!^^' "° inclusive of all the disired 

onl5^Lo Standardised tests are concerned with 

^iL5Snt instruction that are amenable to objective 

^ recognized that local performance is 
o? Jhe^^hi^,"'*^^^"!^"*"?^*' instructional effectiveness 

? ^® '^^y of tbese factors. Among other 

i^SnatfnSI? hiL'^P"^".,"!!^^ environment, ^it ^sV 

educational history, and the quality and adequacy of the 
instructional materials with Chich the staffbas to work. 

stan5lrdl"Jf test!*"^ School Administratorn for one 

At all times, the tests must be considered a means to 
an end and not ends in themselves. These tests have 
5- K.P*"^"*'*?!^ drawing attention of the 

teaching staff and the pupil to those specific aspects 
ot the pupil s development most in need of individual 
attention; in facilitating remedial and individualized 
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inatruction; in identifying those aspects of the vhole 
program of instruction most in need of increased 
emphasis and attention; and in providing the basis for 
more adequate educational guidance of the individual 
pupil. If properly used, the results should motivate 
both teachers and pupils to increased, better-directed 
efforts in both teaching and learning. 

When intelligently used in combination with other 
important types of information, the results obtained 
from these tests should prove very valuable in the 
appraisal of the total program of instruction. Unless 
they are used in conjunction with other information, 
however, they may do serious injustice to many teachers 
and to many well conceived instructional programs. 

KIND S OF TESTS 

Different tests have been developed to meet a variety of 
purpo-es. Some tests are subjective, both as to the matter 
tested and the interpretation of the results- A gtandardized 
test is an objective test that uses the same standards to 
fiieisure and score student performance across the country; 
everyone takes the same test according to the same rules. 

A normed-reference test (NRT) is a standardised test used to 
conipare students* performance in terms of a carefully selected, 
nationally representative group, or norm, on the same ^^st; 
performance is based on total test or subtest scores. < in 
contrast, for some tests, such as the SATs and ACT, the nortr. is 
based on the others taking the test, rather than to a 
standardized national norm.) 

A cri terion-referenced tes t (CRT) differs from a 
normed^eference test primarily in how test scores are 
interpreted and used. A cr iter ion--ref erenced test is used to 
evaluate and report performance in terms of specific 
instructional objectives or skills, stated in measurable terms- 

These labels are not mutually exclusive. Many 

criterion-referenced tests are normed, and many norm- referenced 

tests may be subject to criterion-referenced, content-based 
interpretations 

Teacher-m ade tests generally are intended to provide 
information about individual student's performance JP^^^^; 
classroom-oriented, curricula or specific needs for information 

about students. These tests are f^^e^J^^^^^V^'F MK??cHl<^.n^ 
textbook tests, which are developed by textbook publishers and 
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^BB?SSnr,JC textbooks or be provided to teachers as 
aseSlu?^ f?el(!«nM5"??S^ materials. Both of these tests are 
mea^rrr«t»^f""^ "^'^^ ^'^f*'®^ °" "P°" ""^^Js and help 
T^lfviSuaUzS'T^s^rSctfoS" " ''^^^ ^= facilitate 

testPnLr*rh!^!S * variety of formats. Wultiple-cho iee 

liiffno «LI ^^^J*?^?^®^ °f objectivity and uniformky of 

^2^1^' 2^ administration and scoring, and low cost. 

^f"''*"'?^^! ^° ^"^'^ particularly if they are 

utilised as the exclusive method of assessawnt. 
..Performance-based tests." "authen tic assefisffientn - 
.ri f!!!*^! T!r ^^g5|^^ generally are open-ende^ tests that 
^rr?^ "iays. writing samples 

Pr^cticums. or oral or visual 
i!SSr generally are more expensive. 

lSm?nT«!^«r!i](®' '•e'l^ire more training and preparation to 
a^irm^M^A^ eva uate - factors which Ilso can hake them 

vSli5T?v «nrt^«??^K?i*i '^^ concerns for fairness, 

validity and reliability, standardisation if used for 

a^^rlllld^iith ™u?^?«,i^ f ^" high-stakes situations that 
are raiseo with multiple -choice tests are applicable to 
performance tests. i-t-^iVBi^iB uu 

Performance testing and standardized testiua ace not 
mutually oKClusive. :t is important trp^i" o2t tha?"or 

IZtl ^tlll J2'^"f/"i.^^"«"^"9 asseflments - performance 
f^® been offered by test developers as part of their 
standardized test batteries. Publishers «e now offering 
b!«erTea^* portfolic tests to supplement ?heir currint test 

^t-«nrt^r5<!i^ the particular advantages of a norm-referenced, 

c5u2t?on a^ff^L^^'^ ^ST^** '^^^^^P^^Y ^aUdity in data 

*rf^y«^S' interpretation. It enables evaluation 

™r^^®!JJ '^S^^^f'^'^ in various grades and subjects for the 

re™n*?!^^??^"^ reporting achievement gains in terms 
Q£ a common reporting scale (e.g.. normal curve eguivalent or 
grade equivalent), with nationally representat Ivrnorms U 
?hr«??f.?2 "ther tha?; a fubJective.%2sesImenJ 

greater and the costs of administration are 
of 10) ^ performance assessment {by perhaps, a factor 

^or»»-referenced. standardised tests also enable 
rt«f?^4i f?^ problems in specific skill or subject area 

deficiencies for teacher attention and remediation. This may be 
particularly important in the early grades. 
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Norm- referenced, standardized tests use the same or P^^llel 
test items for aU students, which m&kes scores for all students 
comparable; use of one level per grade facilitates 
criterion-referenced interpretation of results ^o^^^f^^^^' 
buildings, and systems- Individual scores can be "ji^^*^^" 
comparabl4 national norms. One skill can be compared to another 
on a pupil, class, building, or system basis. 

A classroom may have such a wide range of skills that no 
simple test can be equally suited to the entire range of 
achievement; NRTs fof different levels of achievement can be 
S nmerei so that each pupU takes the ^f^^,^^" .""^^ 
most closely to the individual instructional objectives and 
levels of skill development. 

Rpj,E_OFjrHEjrEST_DEVELO^^ 

Test developers adhere to strxct standards, as developed by 
the i^lri?an Psychological Association, the 

Research Association, and the National Council for J^e^urement 
in Education in the Code. of_FaiiJ'esUng_.PrACt^^ 
a copy of which is submitted for inclusion in J®"^"^^. 
Demonstration of reliability and validity also /^^^^^^^ 
to test users, showing that the ""^-^f "^^"/"^^"If l^hntc 
and its appropriateness for groups of f*"®'^®2%l«rod llvllal 
or linguistic backgrounds who are Ukely "."e tested. Several 
books give in-depth, candid reviews of available tests- include 
the Mental Measurement Yearbook, published by the Buros 
^nstHirre'^ MeliVarMeT^^^^^^ while guides and evaluations 

Ire publtshed by the ERIC Clearinghouse on Tests. Measurement . 
and Evaluation and by other organizations. 

Standardized tests generally are professionally developed 

tests dif?r!buted by coLercial test P"^^ ' ^^^"^ .'^^^^Xza. >ons 
be by the publisher, educators or other non-prof i r rganizat ions 
(undpr rovaltv or other forms of compensation), or oy 
goverLlnfal entities alone or in -?P«"^-" -i^P^^^^r.^-' 
(such as under the National Science Foundation s PufaUs....£ 

Initiative ' ) 

The role of the co. t cial test publisher in test 
deve oprient is very eKt^:.=iive. Based on f J^'"". 
variety of sources, including the educational community, .he 
test publisher determines if there is a need for a test and 
whlth^fit win be financially viable, ^j'^ ^^^^.^^t be 
tho affirmative, a decision is made as to the type of rest .a db 
develooed ' e a norm-referenced or criterion-referenced '.^st . 

or ! cSinltton of the two- In -^"^^^n, ^L'^'oer^rmance^ 
the format (e.g.- multiple-choice, true-false, performance 
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Publishers develop guidance materials to assure that the 
final test is administered in accordance vith the 
standardization, and to provide instruction on how the test is 
to be interpreted. Information is also developed and provided 
on the technical characteristics of the test to support its 
reliability and validity. This is done regardless of the format. 

Scores can be reported and evaluated in a multitude of ways, 
for different uses. Rather than trying to describe scoring and 
interpretation in my testimony. 1 am submitting for the recora 
an excerpt from Under strand ijJ3_^hiey«nentJj^^^ 
school AdministrSForsr published by the ERIC Clearinghouse on 
festTT Me'asurement , and Evaluation, on "What Types of Test 
Scores Are There. " 

Much controversy has been generated recently over 
j^orm-referenced testing. To address these concerns. I am 
attaching to this statement several articles from commercial 
test publishers that were included in the Summer 1988 
EducationAl_Mea_sjiremenj^^^^ ^^^^ provide an 

ext ens i v e r e v i ew of t hese issues- 

WHAT SKILLS ARE TESTED? 

Higher order skills, not just basic sKills. can be measured, 
even in a mult iole-choice format, in a standardized test 
(remembering that it was only a very few years ago that 
publishers had to respond to demands for assessment instrument*; 
for the 'back to basics' movement), we recognize, too, that 
there are more direct ways of measuring higher order skills and 
such instruments have been available from test publishers for 
some time. 

As previously stated, the multiple-choice format used in 
assessment instruments has some attractive features. It is an 
efficient and effective way of measuring many educational 
objectives. While we recognise that it has limitations as well, 
it is important to recognize that most measures, including 
criterion-referenced and performance tests, are samples of 
behavior from which inferences can be drawn. For example, a 
multiple-choice mathematics test, which includes five exercises 
in addition of two-digit numbers with carrying, is a sample ot 
all the possible two-digit numbers that we want a student to be 
able to add For efficiency, we chose five exercises, and based 
on the student s performance on those* we infer what the student 
could do if presented with many more. Similarly, we may present 
a situation with several complex problem-solving exercises m a 
multiple-choice format. Based on performance, we can "^^Ke some 
inferences about the student s performance m some of the higher 
order skills m the mathematics area. 
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™.rf«J!!Iii2i^^' infer some important aspects of 

^ltin?«^L!" "^^^^"9 from items conmonly pr^ented in 
multiple-choice language arts tests. 

Neither the pcoble.n-solving nor lanquaqe arts tests are 
tS'i^d'!^ observJion of s^udent^^^rfoflance over 

aitfetent situations in solving problems and in 
producing written material. 

a constant theme of this statauent, that tests 
Sn^,«hL''^ mutually exclusive, I again want to point out that 
publishers of standardized testa cuirently also offer a variety 
of performance tests (including listening skills and writing) as 
well as portfolio programs, in addition to multiple choice tests 

fr,. ^f}^^'<^^^^^Pi^-choice or performance tests, the keywords 
^^^y ^°^^y' are validity and 

reliability. Publishers cannot and should not market a test 

► " ^demonstrated to be valid and reliable. This 

^i"® and money, extensive research and develoranent, 

testing and reworking to assure that the test works. 

RECOMMENDATIOMS AW> SUGGESTIOMS FOR FEDERAL PQLIPV 

i:his°!!B^rf«n4?5 l^^ publishers of standardized tests, 1 welcome 
this opportunity to meet with the Committee and discuss 
standardized tests and our role in the educational process. As 
L!? beginning of my statement, publishers want to be 

part of the solution, not part of the problem. Publishers are 
not simply printers, bookbinders, and marketers. They are an 
^2»?!ori ^^'l educational system, providing an essential 

tti ri J TS^ r®^^ " initiative for and bearing 

tne risk of developing new and innovative materials. Just as 
Congress *MJuld not think of addressing the future of the 
™'!k?TJhH® "w^fli'^ consulting with automobile manufacturers. 
™1«u«S''.,''^?k" '^fn'^in^e be consulted and included in your 

deliberations over the quality of education and tKe 
development and assessment of national education goals. 

What recommendations do we have for Congress? The first is 
that you continue to hold hearings such as this on education 
issues, particularly testing, as a prelude to any possible 
future action. x *~ = ^ 

Congress should continue to assure diversity of 
testing No single test, no single curriculum, no single 

- or should meet our nation s diverse educational 
needs. Competition among test developers, including a vigorous 
private sector, should be encouraged. 
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Third, publishers have a role in making whatever testing 
program that may be adopted by a school or for a national 
education goals program work. They provide an economxcal and 
efficient delivery system for assessments. P\iblishers have 
traditionally served as an important bridge between sound theory 
and sound practice. Indeed, they have been the vehicle for 
getting local school acceptance of new concepts and the 
resulting products, and for enhancing and modifying those 
products as needed They have been the primary lin^ between 
those who create and those who must implement. We do not see a 
change in this role, nor do we believe that a change is 
desirable. For this reason, it is important to involve the 
publishers early in the conceptualization of products resulting 
{rem sound research. 

One of the crucial concerns is the proper interpretation of 
test results. Our fourth recommendation is that Congress 
consider funding for targeted, m-service training to ^^^^^^kL^ 
and administrators in interpreting test results to enable them 
to use tests better to improve instruction, and to convey 
information to students, parents, and the public 

Fifth, state and local education agencies might be required 
to develop a comprehensive assessment plan, which would identify 
instructional and accountability goals and objectives and the 
assessment instruments that would be used to achieve them and 
measure progress. The plan could include specific programs for 
in-service training, public information, and for assuring that 
tests are selected, used, and interpreted appropriately. 

Finally we do not believe that the fedv-al government 
should get into the state and local testing business. We 
commend the efforts to develop national education goals, and 
stand ready to develop and administer the assessment instruments 
required to measure progress toward attainmt.nt ot such goals. 

X would be remiss if I did not point out that while 
publishers are trying to respond to the need to develop 
challenging and innovative tests {parallel efforts are oe.ng 
undertaken by publishers of textbooks and ^^^^"^.^^^^f^^^^^^^J 
materials), federal tax policy is frustrating its achievement- 

The Department of the Treasury is insisting that publishers 
of tests and instructional ^^^t^^^als capitalize^^research^and 




iqh 

risksT'^and^onrieartim^ is shortsighted as a 

matter of educational policy because it discourages the 
Sevetop^ent of the innovative quality tests and textbooks our 
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discriminatory and unjustified tax 

l^rolilitt clnlr^Vt^'' ''^^ AdministrlJTon) to provide 
This CoBmi?t-»f .„ ' outcome remains very uncertain 
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Edward Drahozal 
Rluerside Publishing Company 
and 

David A. Prisble 

The Unlt^er^ty of Iowa 



The authofs. !>ot/i 
affiliated wUh mverslde 
Publishing Comf^any. 
discuss the factors they 
tntnk explain the Lafce 
Woi}egon phenomenon 
and caff for more 
appropriate use of nor- 
maffue comparisons and 
more complete reporting 
of test results^ 



The fijml report of Fnends For 
Education (FFE), "N»lK»i*lly 
Nonned Eiemeuwy Ach*everoeni 
Testily m AmmcB's PubUe Sdwo^'' 
haa appeared in several preliminary 
fcnns< and portions of its contents 
have been discussed is several na* 
Dofial newspaper reports. The f^* 
lowing comments ait based ^oleJy 
on the conceDt of the ftnat report, 
and. thcwgh wt Uke exetptson to 
some of ti» data analysis m^hods 
employed, some of the mt^reu- 
tsoivs oj^ered, and some of xht con- 
clusions neftcbed. these ren^arks 
focus primarily on questms raised 
by U» report. These are lasw* tha: 
complicate the use of agBTVftatcd 
test soores^wuh regard to both 
their formattoR aod their tnterpre- 
unon-to ^nthesiie state versus 
national eomparuont m the face of 
different te^K battenes. different 
standardisation samples and meth- 
ods, diffefvnt years of norming. an<i 
different score scale units for 
reportuig. 

in^e study of state and distnct 
performance reported by FFE ap- 
pears to have been conducted as 
carefully as possible under the cir- 
cumstances. The issues thai the 



report raises— accuracy and com- 
pa/abiitty of norms, currency of 
norms, setectjvity of pupils te&ied 
and reported m reports to the 
public, and temptation lo teach 
spenfic content when educators are 
under accountability pressure— are 
not new. What comes as a genuine, 
unexpected, disappomung shock \a 
the apparmt universal appeai of the 



Edward C DraJUital \$ Sr^uir Pro- 
gram AdmiMtiVlar at TW H\vfrftd$ 

Avmm CkuogfK H Htfptn a^ 
tit praet\eQl Q*pprt» xnurprtiatwn. 
u9$. and ffyorftup of ^itmfniafy and 
Mocm^ry tmu, prtmarUy iUmOttntuMi 

Oav^A FryBtviMAuccxaUFrofft- 
ior amd AMtiStan* Dyrretdr a/ eAf tova, 

Ci;^ iA Ht tpenaitia »» aehuw 
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limpliyk objta^ve of bti&gabov« 
tilt HAdo&ftl ftvcfigv iiKt the extMi 
to which ftdioole M wccrafy in 
mamig^ KNndkiv U) apowr abovv 
th« nahmf tvengv vK«n bcfd 
with pKss&rcs Md ma ultiamtm* 
from poUttdm. pren, the eouita, 
and evm watd^g grcmfM. 

De^tM the ^vof^ommgt that 
can be ctted reonftntmg the mmirt of 
the data that FFG anaiyud and re- 
ported, there ts ample evidmcw to 
warrant dwe examination of the 
gnmp $ ftindam^ttal questton: Why 
an 90 many {wpila or idwds or 
states af^cahng to perfonn above 
the national avmgeT Th* quest^m 
seemt as simple and strat^ht* 
forward at the one posed several 
years ago! Why are tett icores 
dedinii^ firotm year to year? We 
believe that the qtiwstkm raised by 
FPE hvab the seorfrdccline ques* 
tkm in sigTufkaMe and. as was tntt 
of the score^edUM inquiry, this 
sMrch for rBfo)utM>ii is likely to 
yield multipie. cnwofnitant exy£uia> 
trans. There is no single best 
answ^. A doacr examination of the 
issots by PFE. the pubhshers. and 
the state and dis^iet test coor 
dinatort nught enhance our shit^ 
to use test data to ftuther our 
pnmarr god-to iflsprDve the qual^ 
of instntctioii provided in our 
schools. With this purpose in mind, 
the remamto of this papef is de> 
vot«d to identHyinff «4ttt v« brieve 
are the most cruaaj issues and to 
pmMting a scheme that we would 
use to compare the perfttfmanoe of 
Stat* Of district groups w>th na- 
tional pupd or school nom groupa. 

SoM M»iot Isssft 

Aecuracff 4^ ^orma 

National oorms for standardisad 
achievement tcsu ar» baaed on a 
sami^ of po;^ and aehoob (atten« 
dance centers) obtainad through a 
complex, multiftage sarapHog 
scheme. Cach puhhshtf sirivea to 
ensure that the natiottal populatwo 
of puittts and schoda a pnperf y 
repfsseuad m iu noma sample. 
For exan^ to the sfianctefdisatioo 
of the Iowa Tesu of Biaste SIdlKs 
(ITBS) m distTMU were 

chosen oo the basis of geographic 
rqpoo, eoroliment siM. and soao* 
coDQomic chaiBCtefMGSi^ the com* 
munitisascmd ITie staadanfintion 
is ajomt n^KHuft^ til the author^ 

Sttmroer IMS 



publishers, and school pmonnel 
Rigid omditkms for district pur* 
tvipatioo indudad the prmfisiOD f or 
sami^iAg atsefidaae* eraten of the 
di^rict 1^ the pt^&J^KT n^her than 
the school admintoatioii. An 
adn^taie sfts^siing pto is ncocssary 
but sufiRcmt to guarantee ade- 
quate norma. Only if the iAmd la 
realised, only if the SBO^ obtained 
r^Wts the san^ile desired. wtU the 
nomta rspreaent natmal pinn] or 
school ach^«n«)t aeeurstelly. 

To the fxi^t that aoy pubhah^'a 
norm misrq»rtsttit the national 
distribution of pupO and sdtool 
Achievement, comparisons with 
^iher of these nonn gimtpa wiD d^ 
tort the estimated schievvnent levd 
of the group in questioo. An under- 
rmesentatioii of htgh-aehieyii« 
Kboob h^h-achievmg pi^Os wiS 
cause the natimi nonna Co be 
"jfffter" than they ought to be. 
That is* an averagehadttevingpupd 
wtU appear to be above average 
when refneneed to a group whose 
aversge is bdow their ttaorecieil ««> 
'^tme avenge, 

Tlie tai^hiv pUn, itt&irs of the 
obtained sample, wid wilting 
Bch^nes used m the standaRSwkNi 
of eadh adtievcfBcist battery in qiMa> 
tten should be «amined to ^^cer 
mme the re^vaeotativeBeis of the 
IHtbliAhcd norm. Tha should be 
done s epante^ for pi^ and school 

nflfTRSf 

/Seeeucy ofSonrnt 

It » a wtlMoaunemed Cset that 
■chievemcm in gfidfls 3-S has been 
steadily since the tale moa. 
Thm^ the yw^Ofav diffotema 
might be regarded as minor of 
a grade a eu i valent month, on the 
averagei the comulathre cOset mr 
1 0 yean is sgitilSfiast (appTOto^ 
* ly I moQtha. on the everagt)L Ob^ 
viously. those whooon^airsie 1987 
pcrfbrmance ^ their puptit with 
that of other pupOs who were tasted 
in 1978 (nation staadardiatioo) 
wUI be usiiv 'softer^ nonnaaad 
wiB have mora ptQMla appearing to 
be above the uaiioQal avrngrthan 
really are. 

We have pi^fiM fa^bmitfta oo 
changes in stud«tp«Coniiam for 
the past 30 ynra. IMa for idftS to 
19S4 are sumimraad oo pi^ 
148-153 ^ the new fTBS Msmtal 



imts & Hoover. 1986X DiSmncea 
in perfbrmanoe iwy ^ cest« gnde. 
and score toveL Ttm 1977-8S com- 
pottte seora diflimosa are cuiiit^ to 
lune pmnitile ranks (FRs) ai the 
nMdiao io moa grwki, but dif- 
femeaa in laiqRi^ meed 10 PRa 
in sevml gn^ at several score 
levels. 

In periods^ fiuetuatiog achieve- 
ment leveta, the reoMKy of the 
noma is a critical issue. When 
Ach»vefl»nt levels are relatively 
stable ov^ lime. 88 they have tended 
to be at the grade K-2 levels, "old" 
m>rms do not int^eiv with score 
iaterpreta^ms. assunui^ that wt 
have curhculum stability as well. 

JVofurv Vr<sted Fofp^tUUum 

If we have good reason to believe 
that ptquls in a given state should 
have soma, im m avmge. bebw 
the naiiOBal average, we must be 
certain to define tlii ^(^ilation fotr 
which we expect the prediction to 
h^ TWe are sevml rriated 
issues regan^^this point with 
reject to the FPE dM. If State X 
i^MftB amasn notmal curve equiva- 
lent (NCO 46,000 fourth grid- 
era, we should ask these questions: 
How many fourth graters were 
tested but not included tn the com- 
putation msan« and what is 
xh9 nature of the scms of Uxoe 
who were excluded trotn leportmgf 
How many iburth graders are there 
in Sttte X who were not tested and. 
oonssqoent^, who wm not inchxied 
in the rtp^UA acorea? And what 
are the acfaievem«it levels tike for 
theae studcnia who were not tested? 

Based on the Dey>artment of 
Pufaration's Ccnt^ for Education 
Statteiea jSsO 19S& eBraOmente pit), 
ioeted 10 im, the pereentage of 
•tud«ita for whom aoores are re- 
ported in the PFE report vanes 
mm a low of about 85% to mon 
^kan 96% ol total grade ennilbnmta 
for nost stafiie for which ftiU-gTB(to 
tetting ma rqmtedly dnie* (For 
ooe state with pidklic aehooj eurod- 
BiMta 0$ about 48.000 student* per 
grade, aemgts and FBs are re^ 
ported for appmimatefy 37.500 
ftadenta. vhkh isabottt8Q% of the 
tetrt iuroflmenDltte <fiscrepancy 
betwesp the reported stat e sco res 
and the espsctatkms in the FPE re- 
port may be in p«rt due to such dif- 
feienees between tested and uamI 
ettrolled pofRdatums of students srvt 
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spmflcaUy to the naiurt of the por- probably have 
tmn of the stxidcnt populanon not detenruiM^ 
tested. 

Ad^%uiey of Expeetatwf\$ 

Educators have developed some 
expectations about how puptii and 
groups of pupUs might perform on 
achievement testa based on their 
smdy of the relationship of school 
arhjevemenl to other social, poiiti- 
rai. and economy variables. This it 
why we use such vanabies as en* 
rollment sire and soooeconomtc 
status for stratil^ed sampling m 
standardixations. FFE has used 
some of these reianonships m at- 
tempting to develop expectations 
for state-levei and jichool-distnct- 
tevel performance. Per -capita in- 
come, graduation rate, and college 
entrance score averages are among 
the "standard barometer? of excel- 
lence" employed by FFE. Though 
we do not deny the value of these 
indtcaiors as part of the prediction 
equation, we realize that tt is not 
possible to predict achievement m 
this way with high accuracy . For ex- 
ample, the achievement test perfor- 
mance of Iowa pupils \s among the 

very highest in the natwm, yet these 

facts about educational conditions 

m iowa seem mconsxstent with that 

high tevet: Iowa ranks 27lh among 

states m per-puptl expenditure. 

39th m average teacher salary, and 

44th m spending increase from 

FY86 to FY87 
In view of the less than*perfect 

relationships between achievement 

and these other vanaWea, ^trea' 

«on of whatever expectai»>ns about 

achtevement we may formulate 

should be tempered. Thai ts, what 

we are able to say «nth reasonable 

assurance about ho V many pupils or 

schools should aoore above a speofie 

pomt ^the mean in the national 

norms distribution) is not very 

useful. Consequently* we nugbt in- 
stead settle for statementa like 

these for State X: "About 40% of 

the fifth graders tested ahcukl score 

between the 2Sth and the 7Sth per^ 

centales oi) national puml norms," 

or "About 49-V&% of the thini 
graders tested should score above 
the naoonai pupit median <SOth 
percentile)," Of course, the abUtty 
to make such statements depends 
on a far greater understanding of 
(he statisttcal relauonship between 
those variables than most sUtes 

U 



been able to 



Teaching th4 T$$t 

PupOs and their teachers who 
participate ui the staraiardaation of 
an achievement battery )»ve not 
had an importunity to see or iXitdy 
the specific test questioi^ used. 
Thus, havwg no practice on the ^e- 
cif^c test que^ons ts one of the 
stringent preconditHms of the stan- 
dardiiation process. Subsequently 
when these norma are used to intrfr- 
pret the scor^i of pupih who have 
been dnUed with the exact test 
questions, the result is an over- 
representation of the anwunt of 
knowledge and skiU possessed by 
such pupils. Likewise, when the 
scope of the curriculum » narrowed 
to eneonr^kass pnmanly the objec- 
uves m«ured by the exact test 
questjors, the relative standings of 
the pupils wf»> experienced the re^ 
stnctsve program of study will be 
overestunated. 

No publisher condones this use of 
tests, and few teachera pr^iably 
follow such abominable practices. 
Those who do are neariy always 
motivated by signiScant negative 
consequences associated wuh 
scores that might turn out to be 
below expectation (not always 
synonymous with national average). 
Unfortunately, for some educators. 

retention and salary mcrea^es 
are tsed directly to the test scores 
of theu^ pupils. The author* of the 
ITBS have always decried the use 
of aehievemsAt scores for such pur- 
poses and instead have campaigned 
for the use of thm scores to im- 
prove instruction directly. 

If certam tesu ai« to be used 
sthctly for aoeountabilhy purposes, 
their security must be ensured so 
th»» :he scores that result will be 
vaod for that purpose. The dc^lars 
required to ^ssure states and dta- 
thcts that the test forms they will 
use are secure wouhl be far greater 
than the value of the infonnatm 
derived from using the secure 
forma Those dollars w«Ud ttkely 
have greater and more vuxbte iro- 
pacta on teaming tf devoted to 
durect instruction instead. 



Seort Analftis ami InUrpr^atUm 
With which norm group, pupils or 
schools (attendance centers), should 
av erages from State X be con^»red 

Educational Measuremtnc 



to interpret the scores of pupils 
from that siateT With which norm 
group, pupils or schools, should 
averages from Distnet A be com* 
pared? There are only two choices. 
pupUs aiKl schools, because no put>- 
Itsher ^vKies m^ms for »hool 6is- 
tncts or for ««tes. This is a funda- 
mental issue currently facing the 
CouiMnl of Ch»f Suie Schooi Of- 
ficers as they contemplate optional 
for providing for siaie-by state 
achievement compansons m the 
funire. The choice to be nade is not 
a matter ot w^stmal preference but 
a matter of the logKal correspon- 
dence t>etween the units to be com* 
pared. That is. averages of school 
buiklings should not be referenced 
to a distribution of tndivvdual pupii 
scores, dbtnct averages should not 
be referenced to the distnbuaons of 
either xhoo\ building averages or 
pupj scores, ana sute averages 
should not btf refe'-enced to any of 
these three dismtnitions. In view of 
the differences between these sep- 
arate distnbutK»tt. It ts most logical 
10 reference a score or average 
score to its own kind When the 
most logical referencing is not 
posstbie« apprt^mate caution should 
be exercised. 

The nation *! pupil norm Rroup m- 
dudes pupib whose scores on a test 
are as high as perfect tPH - 
those whoae soorea are as low as lero 
or chance average (PR * Ik No 
school (building or attendance 
center) is likely to have an average 
scofv that IS perfect or lero Infact, 
on the ITBS and any other test with 
recent sdwol norms it is reasonable 
to expect that no school will have a 
raw or scale score average higher 
than PR 88 or lower than Pft 12 
compared to ibe pupU distribution. 
Because many school districts are 
smgle-grsde-within-smgle-building 
entitieSt the distribution of school 
dt^nct averages probably would 
encompass the same range as the 
distnbutKm of school bmldmg aver* 
^es. The school district distnbu 
noTK however, is likely to be mar- 
kedly more leptokunic and less 
vanabte than the school average 
distr^mtion. In terms of the pupil 
distribution, the distnbuiion of 
(^strict avenges might range, effec- 
ttvet^, between PR "5 and PR 25 
Finally, mot f of the sute averages 
on a teat for a given grade might 
ureil have actual bounds that corre 
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TABLC 1 

Percenuges of State X Fuptts Performing 
WMIn Seltctett Matlontt PUpil flercentite Intervals 

'•I^ P>r cewuge K ^ 2 } 4 i t 7 B K-d 

10 20 21 24 20 21 20 20 21 19 20 7 

r^-f 15 22 24 21 24 23 2S 24 24 23 23 5 

^-^^ 25 26 29 2S 27 28 28 29 2d 28 27 4 

25 19 18 18 17 16 17 17 17 19 \T t 

10-24 IS 867786767 t.9 

''^ 10 5 2 1 4 3 ) 3 4 4 34 

n«tKm*l m«dt«n 68 74 72 71 72 73 7) 73 70 

Perc»ni»f# be low 

n*iK>nj| .i^ian J2 26 28 29 28 27 27 27 30 28 
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tpmid to PR 60 Md PR 40 on th« 

wcauM nonn» for ^stnet avfir. 
ai«s or for mte mrign trt ix^ 
ft vaiiAbit. districta 8nd ittin oftea 
th« p42|Rl and tehool acmiti that 
do exist . Hlitn 8 diKriet Avmg« is 

should bt thoitght of w tht Mofv 06. 
lAuicd by Kh« avoFM p^qnl in tbt 
distnct We mi^t Smj, for exftm* 
p}«. that th« avm^ pupil in LHs- 
met A seorsd higher than 63% of 
pupils nationftUy. Usng the same 
rtiKmaif sod the entmase given 
above, the avenc* in most 
siRtei ss Q0( likehr to exceed PR 60 
<^ faU bdow PR 40. The velu* of 
fiich mfcNination ts highly quee* 
tionabte. 

A matter related to this genera) 
issue of aoalyeis concerns the 
methods of fiompuiatmal F^aoKon 
used to aggregate and convert 
seem. As an exan^ of the prob- 
lem. « grade 4 schod average GC 
composite sooiv of 42.0 (obtau)ed m 
the tan) «i the [TBS has a PR of 46. 
and e score of 43.0 Hu 8 PR of 83. 
By intemiftCiM and rmndij^ an 
average C£ of 42.6 correramds to 
a PR of 49^ or $0. If GEs are 
mxnded before converting to PRa^ 
a 42. & couM be treated as a PR of 
46 or S3, depending on the rowx^ 
convention adc»ted. Of course. th» 
lUujtration underplays the magni- 
tude of the distention thai could 
result with distributiofls of either 
school district or state averagesi 

FuUisk^Uitr Aeipe«ei6tiUu» 

Though it is in the best interest 
of both publishers and test users to 
have tesu and scores used properly, 
netth^ can ensure that the oth^ 
wiU do iu MTt willingly and un* 
selfishly. Publishers must be 
counted on to sfiaE^ardbe and 
analyse results in profetsteally ac^ 
cepuble mannen. Thqr must guard 
against potentiai mtsuae feqr inionn* 
tng educators of the intended usee 
of the teM Uwy fniblish and vam 
against the poa^>)e misuses that 
might be anticipMd. Publishen 
must do thesr utmoei to provide tcflt 
materials only to those who are at 
least miniinauy qualified to handle 
the testa and Mores in a profes* 
atonal way. State direeton. super- 
mtendnits. teachers, school bottfds, 
and the public, generally, do not 
have the resources to monitor the 
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effertivmss ^ publishers m at- 
^ to these obligittoos. 

Publishers, on the other hand, 
cannot monitor the use of their in> 
^uments effectively to cunail 
mtsappUcatkm. mouse, or misinter- 
pwtation. Often after the ^ a 
P^^>li8her can reeogiuse inaf^tro- 
pnate use-*vbeth«r intaitionat or 
unintmti<mal-and attempt to per- 
suBcto the tts^ to modify a int^Qsal 
or rapon. Sosne school dtsthcta per- 
f(^ excesmve aut^ ts to enstre that 
all students who were to be tested 
tn cad> stt«daooe center were actu- 
ally tested. Some dtsthcta also audft 
refiilta (md rete« suspect groups. 
But for the most part, pubtishers 
ars not aware of and have no ocm- 
trol ov^ school iti^ricts' test ad- 
mmistration omditms, the a^u* 
dents inchtded in summary data 
r^xvtad to the pu^. (m* methods 
used to ^mthme dau to make teat 
results more palatable for leas 
aophiMieated eonsumefiL 

of ost test authors and publiahe» 
go well out of their way to com^ 
wita the standards for educational 
nnd pqrcMogkal tests edited by 
the pioftekm. Test score users— 
taamn. administraton* legisla- 
tors, and c^}^ pobtk grDt^a-tend 
to know far lest than they should 
about the nature of testa or the fm- 
ciplee with which tnt mal»f« in- 
tend f(^ scores to be used We 
shouki not denounce a test because 
a state committee uses the wrong 
norms or incorreet statistical 
analysis ppocedum m reporting. 



Likewise, we shouki not blame 
users for results based on shockly 
staodardisatm) procedures or on in- 
adequate or deceptive descniKJons 
of such procedures. 

Finally, publishers are oblinted 
to diento to mamtain the conSden^ 
tiAlity of test data It has been and 
ahottW continue to be each client's 
deetSKm to release test data and to 
determine the nature of any dau to 
be released. Reporters, citizens, 
atiiens' gmips, and others ivho 
wish to obtain test data should re- 
spect this pul^isher<lient relation- 
ship and saek leloBse ftrxn che &!1uqI 
<totrwt ^ state, dependmg on their 
level of interest and the dictates of 
stale law. 

A Sampk Reputing Method 

We recommend an approach like 
t^ one described below for states 
that wi^ to describe the achieve^ 
ment levels of ttttir pupOs in rrla- 
tion CO pupils in a oatswially repre- 
sentative mm group. £iactly the 
same moeihsrea ooukl be laed «vith 
school butkitag (attendai^ center> 
data. Table i shows national PR 
tardea in the first cohunn and the 
correnmMiing percentages tn the 
second oolomo. ITm body of the 
table shows, aeparstely for each 
gr*de. the percentage of pupib m 
State X that obtained national PRs 
in each range. The last column 
shows the row averages of the per- 
c«itag» values. (Note that these are 
pcfocntages and not peroenole rwiks 
and. consequently, it » acct ptable 
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to svmffe them ) The bottom two 
rowt iDdic»£e. aaia by grsde, 
peroentRge of Afam and beknr 
tl» naci«)»l mediAii. A hiitograin 
with oM diitjibutkm ntpenmpoaed 
on the other or » stm^ bar grmph 
wouid |>rovMie a belpftil vimt dia* 
play of the same tnformabon. 

Itw mam ttivantige <tf this method 
of reporting compared with report- 
ing simply the percentage «conng 
above the nmcsyal vnedan tsobvioua. 
Between-grade differences and stm- 
ilannea can be examined, hut most 
tmponant, diserepai^ieA from the 
natK^nal dutnbutioo can be ac- 
counted for m each of several aeg- 
menta of the diatnhution. If ail we 
know ta that are above t)w na- 
tionai median, we do not know if the 
"enra" 22% are mo»tJy located 
very near the median, moaxiy 
spread through the upper half, or 
mosUy concentrated m the tail. 



Also, we do not k»o« if tiM extra 
22% an shifted from the tower taJ, 
from ihroughwit the lower half, or 
firom ;ust beUnr the median 

Stfany diatricu uat a reporting 
procedure similar to the reporting 
scheme described above. We rcoom- 
iTwnd that !mch tabular daU be 
supplemented with at leait the 
foliowusg aorta of tnlormation; 
tesiu^ date, teat form and teve^i) 
used, type and date of the norm 
U9ed. and percentage of eligible 
snutents tested. 

Riveraide PuhUihing Company 
and ita representatives <k> not 
believe that the avenge pupU tn 
every gtate has scores above the na- 
tional median w> ti» ITBS. We are 
con^dent m mir stwidardisation 
pmeduiTS and have sufc^ectad those 
procedures to public scrutiny m 
detail in the Manitai for School Ad- 
mtnutrtUon. We have updated our 



norma at least every 7 years and. 
when achsev^nent showed a pat- 
tern of iiwrease m the cariy l$80s. 
new norms were oMained e^'en 
though new test forms were not in- 
trodt^d. We are making plans to 
provide annual national norms up- 
dates for next year. Our manuals 
cauoon users about approprute use 
of norm groups for vailing pur- 
poses. (Xtr hope is that the issues 
ratsed above wUt cause FFB and 
state and distrtct test coordinators 
to rrassGS their analysa and rvpon- 
mg procedures to ensure thai con- 
chisiotts rmhed are based on a 
valid f(Hinda&on rather than data of 
questxinabie ongm and mampula&on. 
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A Response to John J. Cannell 



Joanne M. Lenkc and John M. Keene 
The PsychologtcaL Corporation 



Tiuo represerlatiucs of 
The P^ychotogicat Cor- 
poration present *hetr 
reactions to the Canneii 
report an<S cait for better 
expiMnattons for the 
pubUc of the meaning 
and Hmits of norm- 
referenced scores^ 



!n recent years, pi^ sOeotsoo haa 
focused on standaf^ttxed achieve- 
(TMt test resutta. 'Hiese resulu, 
which are intended to describe the 
perfonnanet of i&dhhduais in rela- 
tion to one anc^bcr. are now often 
used to dsaeribe the perf onnaooe (rf 
groupa of stud^ts. In a rtwi en- 
titkd "Nasiimally Nomed Etemen- 
ury Ai±ievciiisot TfSiifV tn Amtf'- 
tca^s ?yMc Schools: How All 50 
StatM Aiv Aboee the Naliaial Avei^ 
age/' John Jaeob CattneU atl«mpu 
so CMt doubt cA the vaiklitv of the 
inf ormaSMm being V9p^%e6 to de- 
scribe the ach^vsoient of ftudenu 
as aggTCtfftted al the state and/or 
distnct level The report itaua, 
'*11iese tests alkHs all the states t^ 
ciatm tD be sbQ«e the ftatxmai sver- 
a^t the tests. . .altow 90% of the 
aehooi diATtftt ^ tte S^*'^ 
to daun to be above average. More 
than 70* of the studenU tested na- 
tionwide are told they are perfoffiA- 



above the national average." 
In response to Cannell. ^t is fair 
to say that many states and school 
districts report above^average per- 
formance m reading, mathematics, 
and/or language tn the elementary 
grwies. We do not beheve that this 
IS an attempt to misrepresent stij- 
denta* achievement tn the nauon's 
schools. Let us examine three very 



Joanna M LnkM %m V%ci Prm^ 
Msa iMi wm nL ol Tim fgydtofeigMBi Cor- 
peroAov. SSS Aaadewac CMi«t. Sam As 
tamo. TX TttO^-MB 54e fp«wiu« %^ 

and <«ta 
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imporuni issues rvJat*d to the in 
tefpreutjon of ihm mformsti<ifi. 
Ui group performance rpJauve to a 
naticfiaJ norm, (b)!oca} perfor- 
mance relative to ntttonai perfor- 
mance. ami (c)lhe stability of 
ac^ievemciji test norms over ume 

InUrpTf ting Groop Performaoce 
KcUti%t to A NalAona) Norm 

When a test is »iAralardued, or 
nofmed. the lest is lyptcaiJy adrmn- 
tsiered to hundrefis of th<H»ands oi 
stu<Jenis nationwide This normmg 
sample is drawn to reflea specified 
demographic charactenstics of chil- 
dren aitefwiing sf hooi in the Uruted 
States. Such demographic char- 
acienstics include socioeconomic 
suius. ethmciiy. region of the coun- 
try, and uxe of school disinct 
Percentile ranks are then derived 
from frequency distnbutjons of in 
dividual students' scores at each 
grwde. Norms provide a mechamsm 
for descnting a student's perfor- 
mance relative to that of other stu- 
dents m the same grade from across 
the country at a p&rticuiar point m 
time. 

The use of these norms to descnbe 
group performance must be mier- 
preied careftiliy. For example, if a 
sute's average score m readmg la 
at the 54th percentile, the proper m- 
ierpreiat«)n of this score i& that th« 
average, or typical, it^dtni m the 
state performed better than 54* of 
the normir^ sample. It is not ap- 
propriate to conclude that ail stu- 
denu in the state are above average 
m reading, that the state aaa whole 
IS above averagv m reading reittiva 
to other states, or that the state as 
a wh<4e is above arengem leading 
relative to the Rat^nii norm. 

The a^>roach used by aomt siata 
and school disthcta ID the rafMMting 
of group performance it to report 
the percen Cages of students fcofing. 
My. "ai or above the fWi percen- 
tile." or ^' in the avenige and above> 
average range." Although thia 
method of reporting is a|q>ropnate 
because it maustains the relation* 
ship between individuaJ perfor- 
mance and the natioitat norms, the 
reported pereeoiagce shouM be ac^ 
eonpanied by oorreipondiiy per- 
centages for the natioftal normo^ 
«unpte. AHhovgh it ia obvious^ the 
case that 50% of the natkmal sam* 
pie of sttidenta seored at or above 
the national median at the ome the 
Sttamtr IMg 



t«t was standardued. it may mjt be 
the case that 50* of the national 
sample scored at or above the na- 
tional mean raw score or national 
mean scaled score, if the repomnir 
metnc is something other than the 
percentage of students sconngat or 
above the national median, the ap. 
pJtjpfwte natkMttI compamon dJoiii 
be provided so that prt^r infer 
cnces about the data can be made. 

Interpreting Loral Performance 
Relative to National Ptrformaoee 

It IS udikely that the d^nograf^ 
charactenstics of the students m 
any state or school distrvt nunor 
those of the nation as a whole; it ts 
e<|ualJy unlikely that the curriculum 
of any state or local district is as 
diverse as that of the nation as a 
whole. Furthermore, it is not neoes- 
sanly the case that the guidelines 
!iei forth by the test pui^isher with 
regard to the testing of hamii* 
capped or Umit^d Ei^Sb-profkient 
students in a normtng program are 
the same as those used tn actuai 
practice. If there were a state or 
dtstnct whose demographic charac* 
tens tics matched those of the na- 
tion, whose curriculum was as 
diverse as that of the nation as a 
v/hole. and whose admimstratjon 
guidelines and procedtires were 
consistent with those used by the 
publisher for the nmning sample, 
one would expeet the average stu* 
dent m the group to score at about 
the SOth percent^. To the extent 
that differenm exist, we must re- 
mind ourselves that wl^n local 
group summary scores are ime^ 
preted in reflemtce to a national 
norm, the interpmation has to be 
V^aeed in the proper context, simf^ 
that of the group's aver^ student 
relative to the national norm. 
Because it is unlikely thai ti^ 
students tested tn any given state 
or dtstnct are typical of the nation 
in aU respects, it would be uBrcasoo* 
able to expect my pro*]? u» be Of the 
national avertg*. 

Test purchasers, disthcti as well 
ns state agency oft^ s^eet tests 
thfOHgh a process that examines the 
match between the test cotttem Md 
the local cumcuhmt tn mangr caMSp 
the seleecid test is the ooe that best 
reflects the local eurrieuhtm. Test 
users sheeting tests on this btt^ 
may have so advantage wtr the 
nom group beotttse the test is IStsiy 



to be more valid for assessing per- 
fonnance m the local cumculum 
than It IS for asses&ing the perfor- 
mance of a national sample of stu- 
dents being exposed to different 
cumculums. presumal^y having 
somewhat differvnt emphases. 

The Stability of Achievement 
^eat Norma Ov«r list 

CiinmjU-s report suggests thai the 
use ol 'old" norms is partially 
responsible for h^ achievement 
lest scores. Presently, test publish- 
efs PT'-Jm^* editions of their 
tests a 7- o-9-year c^le, and cur- 
rent norms are provided with each 
new edition Because lest adoption 
cydes do not necesaarUy ««ncide 
with test revi««i ^les, it ts con- 
ceivable that the mmns for a newly 
atopted test may be 2 or more years 
old. inerefore, it iscntkally unpor- 
tant that emptricai norming dates 
accompany the reporting of achieve* 
ment test results. 

It IS very enoouraging to note that 
today's students are performing 
better than their counterparts did 
m the late 1970s and earfy I9g0s. 
Evidence of this miprovement in 
performance can be found not only 
from research that test publmhera 
have conducted in equating newly 
published tests to previous editions, 
but also from a recent researrh 
study cmKiucted by The Psycholog 
teal Corporal with the current 
e<fiiKmoftheSta>t^Ac*iewmeiif 
T$$l Series. First standardised m 
the 1981-^ v^tool year, the Stan- 
ford Ser^ was admmistered to a 
nstKmaUy nnrMentative sample of 
350.000 stuilenta ^ spring and tail 
im Tl!ie sample wss Airther stvat^ 
fied actttfding to "user" and "non- 
user'' groups, where "users" were 
(ieflned u school distncu that had 
been using Ehe Stanford in one or 
mer ejpiadea for at least one ywtn 
TfretrnlitilHteiiisnriTBteiniic erarim 
nients. The results of this study re- 
vsated that *^is»a" ouqmrformed 
'"noftttssr^" atti more importantly, 
that *'nooassrs" pff&rmed better 
than die ortginsi oonmng ssmple in 
m s them a t ict, reading, and the tan* 
gwge arts in the ememsfy gndes. 
Two tmpoftast graeralisatKMis can 
be made from this retesreh. First, 
test soores do t^ to increase when 
the same test serte is toed vear 
after year. How e v er, this shoidd not 
nccenar^r beatSfibuied to "teach- 
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to ^ fitit", mOmr. i«tt 
rmUtt prov^ % rmdtd focui on 
areaa in nc^d of ijn(»T>v«nMnt. S«e- 

impfov* ffw\ u> 1986 in lofne 
^)9Ci AMI in tht eltmenury 
ichool gfvdc*. llMfvfore. moiv cur- 
rent norm for tht Slonfonl Sene* 
havf bMn t^vtloptd Mod w ftvul- 
abif to ui»rt of tht bat*4»ry. 

Ilhinng Uus nmt of educitional 
imprDvemenU tt ts importuii not to 
ioec sight of th« fact that use of the 
sMn« nom* over a penod of )re«r> 
eittbiei the (est user U> demonsfjate 
uriprovemem reUove to ft oonatAnt 
reference group. Even if it were 



ceonoiTmlly feanbie for tett pub- 
tuiherf to preduce rrpmentAtive 
utkmol Qormt mor* often, 
ouently updstcd norm rmntnt » 
' movmg taniM," «hm ethcitiocttt 
gmtnt <or tot»ei) wouM be reasked 
by the retatiire n»ture of t)» infor* 
matMMi. tW level ^ achimnieflt of 
ttudenta in the U mted States fm ut- 
creucd in recent yw«, and eduea- 
tOTi must have the ^ipntumty to 
demottftrate thete giutt m oider to 
eostfe the neeeswry support of the 
kxal community m improving the 
quality of eduction. The education 
of yming peopte muR continue lo 
improve, and norm-referenced 



ach»vement letu are ujwful toob 
lo thti endeavor. 

Coaciailea 

Beeaase the public is expecting 
norm-referenced scores to rvpre- 
sefiS standards of performance, tre. 
aa ediicatoet, mutt asftst the public 
m becoming better informed about 
the mterpreiatton of lest results. 
Nstmat nwTnsnve data provide ex • 
trendy important information for 
making sound educatHuvU decisions, 
degree to which these decisions 
are defenstbte depends on a dear 
understandif^ of the strengths and 
timitatK>nB of the data. 



The Time-Bound Nature of 
Norms: Understandings and 
Misunderstandings 



Paul L. Williams 
CTB/McOrauhtiin 



Presenting a view from 
CTB^McQrawmit. the 
author discusses the 
tlme't>ouniS nature of test 
norms and argues that 
the phenomenon of most 
etementary students' 
scoring a^ot^ at^rages 
from previous years' 
norms is a result of 
generatty increasing 
levels of achievement 



Recent interest in the topic of the 
tjme-bound nature of Qormed soc»«s 
has resulted* m pafV atie^ 
turns made in a report isaued by the 
Frwnds for Eduartwm. The key ele- 
ment of the argument put forth in 
the Fnends for Edt^stion report is 
that too many studesti appear to 
exceed tl^ nationaJ avmge. Data 
have been presented in the rrport 
whfch ore saKl to show that more 
sutes and schoo) distncta are acor* 
mg above average than one might 
tntcuily expect. 

!t » an interesting phcnoownon 
that It is through the vehide of the 
Fnends for EducaSKin report that 
the tune-bcHind nature ^ nonna haa 
received some measure of ^ibUc at- 
tention. The fact that norms have 
always been referenced to the year 
of test standaithzaoon tt something 
that has been so univeri^y known 
and understood by testing pfofes* 
«ionaia that )t has not had s ^^r^ 
measure of sttenoon focused on it. 



Perhaps that wt)l prove lo be an im« 
poftant smguiar contntmDon of thi^ 
issue of Edtttatumai Mrfuur^f^^t 
issues and Fracttct. 

' 09 Cyclical Nstitrt of Test and 
Jul IS Devekopmest 

The evotttticn of norm-rtferenced 
testa (SKT%) S5 vsluable sssessment 
inttruments has been characterized 
by the expansion of the purposes for 
te^mg. in the eaHier versions of 
NKTs <fft the mid t960s to the 
mid<t970s). the primary purpose 
was to pf >v>de accurate normauve 
scores so chat group and tndivuluai 
OHnpansons couid be made toa na- 
tional profUe of achievement. Using 
this tnformanon. school admtnis- 



ts 



Rtuarck and Mtmrt^ntni ox iTW 
Educs&onst Mf«aurrmtnt *nd F'^^iir* 
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msbc and tndjviduA) strmgtKs ami 
structioRai uitemnoon «Ad 

achmvffment eould be mooisared. 

An expsftAtton ot xhev pu/posM 
look pitce with th« publicatum of 
iht Cait/'oniia A^hifvemfiU Tmt 
tCAT), F'frm* C and D (CTB/ 
McOraw Hill. Id77>. This bax- 
Xvry. for th« first lime, aj^owed 
seor^ for trmrvctumsJ objectives to 
be reported from an NRT for in- 
dividual 4>XAmm#e9. Although 
t^aHt^r NRT te$l versions did allow 
test admtnistT»icr» to use item 
;ina]yses for minunaJ diapxMnc pur- 
poses. CAT C and D prov^ided spe- 
cifhr instruction^ objective scores 
for the purpose uf more mdividuai* 
ited mstructionAJ pJanmng. 

The ^hedule for the publication 
of norm referenced testa has fol- 
lowed a tmsit, industrywide cyde of 
between 5 and 8 years for the sAme 
(est senes. In tne instance where a 
test company haa more than one 
NRT senea. such as CAT and the 
< prth^n^ w TtMt 0^ Basu^ SkUU 
iCTBi^y publication (S sLa^^g»^red so 
that tme test of the a«nes is pub- 
lished about every 3 or 4 yean. 

This oyrle has been dictated by 
several factors. The first factor has 
J*eei> the 5pt*d with which cumc- 
ular ch«n/fes take place in the na- 
tion's !ichuols NKTs anr (ksigned to 
reflect (he predominant achieve* 
ment uutcom«9 »jhJ currtcuiar 
trends m the natiotf's schools. When 
a new form of an NHT is developed, 
t'ontent iNmsidergttons jUV of pisra* 
mount importaw?- Although cur- 
nruiar trends have a rnsjor impact 
on the content of NRTs, theM 
trends do not ch»i\ge so fast m th« 
schools that more frequent revi^ 
MQTXH of a t«st sertes would be 
;u»tt/Wd based solely on them, 

M the c»me an NRT is revised, 
the cullertion of dat4 for the genera- 
tion of new nat)ona^ norms takes 
p^ace Using a national probabiltCy 
^iampie, data are coHected for 
several hundred carefully selected 
whool dictncts aiid hundreds o( 
thousands of students Based on 
th>» cacffuity selected stratified 
san^iie. nofmatiif e soxres are devel' 
*>ped 

Karh f»f the derved scores that 
Summer tVtM 



em«fg« from tht standardisation 
proeesa, including percentile ranks, 
grade eqiuvaitnU. and fsonnal 
curr« eqwvatents (T^CEsK Km a 
predefined relattonthip to th# 
charactenittci of the nonn group. 
Thus* at the tune the tc^ a normed. 
S0% of the eaaminees w\ll exceed 
the 50th percentile and the same 
percentage will fail below the 50th 
percentile. Denved score tables for 
the test tumefy are prodimd. and 
all sconng of student tests is refer- 
enced to these tables until the bat' 
Vijy 11 either revised or, in rar^ 
instances, when it is rem^med %v}th 
no change in the content of the test. 

Data from naPonaJ probability 
samples are not umalty cot^cted f<^ 
a test mors often than every S to 8 
years because it iS impractical and 
economically mfeastble to do. It 
would not be reasonable to ask or 
expect schools to administer tests to 
large numbers of students every 
school year mtrnkcrtodevelof) year* 
ly norms based on a national prob* 
ability sample. The cost of such 
testing would have u> be passed on 
by the pubhsher to the schools and 
would add substantiaiiy to the cnst 
of school tesDng programs. 

In summary, most targe test 
publishers follow the common and 
decades^ld irrduslry pracoce of 
revising and standanLnng thetr 
achjeven^nt tests about every H 
years. The content is updated to 
reflect current cumcula and in- 
structional practices. sAd new 
norms are developrd so that the test 
reflecta levels of vhievement that 
prevail dunng the school year oi 
whKh the test ts stafMiardized. The 
dates of standardisaciM are gtv^n 
wide publtnty. and all fmrchasers of 
the c«st are awair of these dat«s 

Proper Ister^taiioas of 
Nattoaat Norms 

Because norm- referenced tests 
are not normed yearly on a nabonal 
probability sample, changes m na- 
tional achievement between the 
nonnmg years wil! be reOected m 
the norm scores for groups of 
students. For example* if national 
achievement levels decrease be^- 
tween normmgs, as they did from 
the late* 1960s to the mid* 1970s, 
students' norm-referenced scores 
w>il decrea^te. and more students 
^tii tali \ir\ow the nvedian f5Uth 



percentile) score established when 
the test was ixrrmed. On the other 
hand, when naoonai achievement 
lev^ (iKraie between norming*. 
n)ore students will exceed the me- 
dian establisbed when the test was 
onginatly normed. Rcgantos of 
the duectKH) of natNma] achieve^ 
ment trends, when a test is 
normed. exactly half ttf the students 
will faa above and half wtU fall 
below the newly established 
median. 

At this time« national achieve* 
meiu indtcators ail point to the fact 
that student achievement is gen- 
erally on the mcf^ease. This mmase 
IS documented by the National 
Assessment of EducationaJ Prog' 
ress tNA£?); the Scholastic Ap- 
titude Test tSAT) re^ta; two 
Congressional Budget Office 
reports (l9Se, 1997); and data col- 
lected dunng recent t*st normmgs 
by CTai«cCriw-Hai{l^, m7. 

Thus, dunng a tune of tiKreasing 
n^)onal achievement, the students* 
normed test scores will rrse be- 
tween norTTUDg penods. More stu* 
denu wUI score above the median 
score established dunng normmg 
than will fall twtow n This con/vms 
the sensitivity of the test norms to 
changes in achievement, one t^f the 
tests' pnmary functions These 
normed test scores are vahdl 
measures of student growth 
Although the irference year for the 
scores will be pnor to the year m 
which the test scores are reported, 
the test scores provide accurate 
program and student infurmation 
The fact that the norm vt»res 
themselves refer to norming that 
took place dunng an ea/her year m 
no way compromises the major pur- 
poses for admmistenntf an NRT <»r 
the usefulness of the scones for prt>- 
j^rurt evaluation, student mstruc- 
tional planning, or the mnmtonng 
of tongrtudinal trends HTienrnter 
pretmg the scores, (he i4>st K^ser 
must simply be aware uf the year 
that the testa were norm««d a/>d the 
general direction of national 
achievement trends Interpretive 
guidettnes are found m relevsAt 
tese-retated matenals produced by 
most pubbshers 

T^ie Pnends for Educatittn report 
nas received attentMm pnmAnly as 
a restilt of its ifnprf»prf tftt«Tj>rf ti 
tions of score (hstnbutiuns fi*r 
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ftlsrsyB provided. 

The second exaniple rpi»u^ to the 
Annual Natwna) Normative Trend 
Dau tNTD) published by CTB/ 
MfCrew HtU. Research on this proj 
ect began m 1984. wh«n an emerg- 
ing cu^omer need wa$ tde^ntiiM by 
the company Customer comment* 
abuut the desirabihty of obtaining 
more recent normative data were 
noted m market research efforts. 
Stfch data rtnitd be used to amplify 
the standardization norms and pro- 
«d« a more complete picture on the 
progress local school distncia were 
making in their mstructjona) ^t- 
forts Aflfr 3 years of research, the 
NTD service was offered to CTB 
cu&toitier5. Score repons have been 
made avaUabte on an annual b^is. 
fur the rtandardoaiion year a» weU 
as for the most recent normmg. 
This 5€rvTce is a response to those 
educators who have been concerned 
about the time-bound nature uf 
norm- referenced scores. 

The teat companies do thetr best, 
through many vehtcles. to assist the 
te»t consumer in being a responsi- 
ble user of test lesults. Indeed, 
reasonable testing programs, effec- 
tively imptemenied. a» one of the 
reasons that tthievement is tncreos' 
mg and that we are not curreniiy m 
the dedtne phase that mantfeuted 



ttsetf m the late I9^s to the 
mid- 1970s 

The assertion that scores are on 
the increase does have ment 
Perhaps the p<»4tive side of this 
phenomenon shouid be stressed 
more States and local school 
districts have committed con- 
siderable resources to improving 
the achievement levets of their 
students. AH indicators of student 
ochieven^t appear to converge gn 
this fact, particularly for the 
elementary grades. The Amervan 
public should be gratified that 
achievement i$ increasing, 

t'annell n9»T) charge* thai "inac- 
curate initial norms and teaching 
the test.'* rather than imoroved 
achievement, are reasons lor im 
prov mg scores on nationally 
normed test* The proolem with 
these allegations is that there is bt- 
tie. if any, evidence to support 
them. To the contrary, the body of 
independent evidence suggests that 
test ftorms provide a valid and 
useful refererce m both the norm- 
ing year and m subsequent years 
aiut that achievement at the 
elemeniarv levei has been mcreas- 
mg. If indeed there exsst instances 
of abi'se of test norms and of 
misunderstandmg of thetr meaning 
by educators or the public m 



general, then the proper remedy 
shoutil be to correct ^hose mstances 
rather than to make rash allega- 
tions about the adequacy of test 
norms or queslionabte teaching by 
educators 
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Wftsnuiiftoii. DC Author 
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ttwmds Waihii^ton. DC Author 
CTB/McOrSw Hill H^TT) i ttixfuma 
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V Monterey. CA Author 
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terey. CA Author 
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SRA Response to Canneirs Article 

Audrey L. Qualls-Paync 
Science Research Associates 



The author defends S/M s 
norms, discusses same of 
the difflcutttes in pursu- 
ing 0r. Cannett*s pro- 
posMiSr Mid points out 
that we need to monltctr 
not Just student achletfe- 
ment letxis but^atso 
frciids in curriculum. 

SoaoMr tW 



Science Research A»ociat« (SRA) 
fMogmses the concentt ex|»«swd 
in John CanneU's aiticte. ^NaDonallty 
Normed Achtevement Terting in 
Amencaa Pubhc Schoob: How AU 
50 Statet Are Above the National 
Average/' We di^. however, in 
otu* anesam^t of the tHuation and 
the proposed alteniftthrer Mtof^- 
mg to the artkle. mnt lehoob in 
the naiktfj perfofm at or above avei^ 
age on oofiimerdatly anttoble tfits. 

(bwtog. a» noted by Dr. CenfieH. 
It not oonsttteot with suti rti c el 



theory, which says that half the stu* 
denta shotdd be above and half bek>w. 
Dr CanncU egpreaaee the opmion 
that this tmsxtfistent itacistical ^ 
THMn^m resutU from usmg okter 
te^ okd^r norma . learfi mg to the 
teet, itafttrlfa^ maoipuiation of the 
data pditt^m. eaehiding speoat 

moojv StfWKe Ratarrk Atnonatet. 
far. tSS N. Wadm Or Chttaga. IL 

21 
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ta^ of gmiQ) «vcr«ge9. Aod inae- 
cur»tc n^tm. He goes on to sug- 
gvft ihftt these pnit^ems cu 
c^Bmnsiad by U90 o# on* iMiuare- 
ment t«8t m •ehools AcroAi Che 
ctttmry wtth the fioncun^t <kv^op» 
mentof armuftj ooim^ Our purpose 
IS to exAinme Dr CanneU's condU' 
stonf 4/)d (lOer ait^mattves to somv 
i>f the uoui^ rM«d in Ms report. 

SRA's fiatma) norms are reliable 
ftiKt accurate indjcaton of national 
stvdent pprf<mnance at the time of 
stamlardaation. The charge of su- 
tisncai mjvupuiation of data ap- 
pear to result from Dr iTaaneirs 
apparent misunderstanding of the 
purpoae of the vanaus types of test 
sfcores and fuht^roup norms Schot^ 
mstf wish to compare thetr students' 
performance with, m addition to 
that of the nabonal grvHip. that of 
gmips more ^milar m stnicture 
and student oompost^on. For exam- 
ple, a nonimbbc srhoot may want to 
compare thetr stiKfents' perfor- 
mftivce «nih that of st\<dents from 
other nonpubite achoois The vanous 
test scores, in addition to status 
SQOKS it^., perantdes and stafunes)^ 
are offered to meet the many needs 
of customerVw Normal curve 
eqtiivaieDts (NCE^ are f«guif«d for 
Chapter I program evaluation. To 
assess longitudinai growth and 
determine functicna/ tevets. devdop* 
mental scores, for example, stan- 
dard scores and grade equjyatenta. 



are noeded. 

O Cannttl's aft«matie« to tl^ 
vanous standardised achtevement 
te^ a a naooaal adutvemMit test, 
which nrouJd require a( least two 
major actaona. Tint, this national 
achievement um would have to he 
nom^ anmiaUy wtth a representa- 
tive gmip of Auctotts to havt ytmiiy 
norms. Second, new test forma 
woukl be needed for each admims* 
tratjon to eitminate posutte prob- 
lems of teaching to tii test and test 
security. 

A project of this magnitude and 
complexity would be very difficult 
fogisaeaily and very costiy. Two 
m^pr togistie problems w^ild be 
U>obtaming cumcuiar consensus 
on the test content and (b) obtain* 
ing or mandatuif national par* 
ticipanon. 

1/yeaHy new fcms are nc» an op- 
Uon but annual normmg is, and tf 
there truJy ts a sutmtantiaf amount 
of teaching to the test, the problems 
noted in D^. Cannell*s analysis may 
go away, tf new forms of 
achievement testa art developed 
each yw, therti>y mcreasing test 
secunty. the need for annual norms 
dmunishet sgnifieantiy. Based on 
Dr CanncU*s analysif rrom Khoots 
with tsght test security and titer- 
ature on student growth, drastic 
shifts m student performance from 
one year to (he next are rare. From 
a psychomecnc point of view, new 
norms are needed oniy when there 



ts a s«jr*V^^^ school cur- 

ncuhim and^or student {)erfonnanGe. 

As oppimd to developing and 
nandai^dnuig new forms liach year, 
a mechanism is neieded to monitor 
changes in school cumculum and 
student performance, (Vb« never 
there is a change m either cur- 
riculum emphans or achievement 
levels, new test forms sht^ be 
developed and standardized If th« 
change is str%ctjy a shift vn stt^ent 
achievement, npnonmng ts r^t^red 
As a publisher, we must base *>ur 
decision ^ when to m-w 
forma/rww norms on a systematic 
monitoring system. 

There are several ways to moni- 
tor student progress One way to 
accurately spot when significant 
changes are taking ptace ta to track 
student achievement <m a reguJar 
basis (i.e., annually}. The entire user 
group coubl be used for this pur- 
pose. IV nwPutorevT pi^ooess should 
be capable of producing user-based 
norms« which can then be made 
available to ail customers as an up* 
tsonal senoce in addition to the na- 
tional norms 

There IS at least one ma>or prob- 
lerr with the user-based monttonng 
system, if the user sample is biased 
and unrepiwnta&ve of the naix^ 
student population, significant 
changes noted m the ufier sample 
may not tnily reflect changes at the 
nationai fevel. One way to resolve 
this problem would be to select a 
subset of schools from the user 
group and use it to monitor changes 
m cumculum and student achieve- 
ment annually The selected schools 
should be representative of the na- 
tional p^otaDon of schools with 
respect to geographK region and 
raciai/ethmc and socioeconomic 
status. Once a set of schools ts 
selected for this purpose, studenta 
rn these schools can be tested on an 
annual bass and ncms esn be (^el- 
oped. As in the previous nwthod. 
annual nonns will be made available 
to customers as an optional service 
Because of the representaDveress 
of the schools selected for rnonitor- 
tn$, one can, with a high degree of 
co^i&ience. generalise rrsults from 
this set of sdtools to the U S. pop- 
ulaCHm of schools, 

Becttise SRA rfoogmied the v^ue 
of a monitonng system, we are 
already m the developmental stages 
of implemenUng such a pnf^r^tn. 
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MIDDLE CITIES EDUCATION ASSOCIATION 



Art. ^-V(r»- s»««»» 

Mt«v OM* tNte M *»M 



March 19, 1991 



The Honorable Dale Kil^o, Chairoan 
SutxroB&itt&e on EleTOntary. SacanOary 

aiMS Vocational Echication 
120 Cannon ^ildinq 
Washinqton. D.C. 20515 

Doar Congressman KlidM; 

Pleao© aecf^pt thin letter re<)arair)q the dovelopmont of 
a rvational ass»s«»ont ftyatcm a^ Input froo th« «tddl<» 
cities Education Asoociation foT ths record of the 
Karch 14, 1991. Oversight Hearing. 

The ttidaic Citi«ii Education Association {MCEA) is a 
consort iua of 2B central city school districts in 
Michigan which Sorv© 300,000 {2o%) of tho stiKients \r\ 
the Stats. Th» organ trat ion works to cremate a better 
understanding and »upport for the noods of urban school 
districts and tho many oconooically and oducationally 
disadvantaged c^'ildren w serve. 

While It 18 clear that there l» renewed int«»ront in and 
noveioent toward a national testing aysteo. we tnyliev»» 
there are a number of issues which need to be 
addressed. The following insv^efi were discussed 
recently by the Pirectore of Testing and fc>^aluatian m 
the «CEA school districts: 

* Any effort toward » national auGessioent should 
have a clearly defined purpose. Vfo are uncert^wh 
whether a naticn<sl test is being proposed in 
order to provide our nation's students w»th 
essentia) knowledge and skills, or whether the 
movftioent »» »ore political in nature. We woulrf 
hope that the need to increase stuttont 
perfonoance by isprovinq rurrtculum »r\d 
inst rvic-t ion is the driving force behind national 
test ing, 

* Frior to adaption «f a national assessaient . a 
veil devt^loped national curriculuia UtiilBe^'OJK 
needs to be dovt?loped so that teaching and 
itarning for ail children in based on a coiWROn 
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i*<»t oi cieariy t>tat«»^ 9ualt}. outt:o»c& and «tAnk<Ar»lu. Wtthw* 
broad cumculuo} framework which d€>si*r lb«^s s.tud«>nt outctjWH" 
t o be achioved. stdtos, school districted, &<*houls unJ t »'«»t-»ii*f •. 
j*bm4ld fre free to crcativoly employ a variedly of instructional 
tct rat pgirs and matorlald to imoure students achieve thi?.. 
outcnmes. Deveiopstont and us«* of ah asBoeamc^nt without tirf<:t 
hdvsn9 this currlrulua frawworK vouid be iseasurinq appi«*t: »^n'i 
c^rdncfCu. An ©Vfc^rypupil. nationwide t,«e»t is iJ>oaninqler.t"' it 
what iR l>oinq tau9ht dalters wideiy. in other worda. « tevt 
whiv'h ji>easureB procfrcia tcw 4rd previously fipocitied c'<wt»>4 n 
s|o«ils and stud**nt out<-nmcB woul<1 i^f^^n J I ^nd v,"^uKl fi.'t 
f>ev'es5i.fr 1 ly r.tlfJe Iti^al cr<-.itivity m di'vclnpmenr oT ? ii. 
f9C<S{l? rcjch thost? oatcuToet*. 

At?/ ru^tionfil tff.t :;hftuld haui'd on a rjvh. wh j 1 1 <'n<i j rt ; 

Wc« .iri" v'onr«»ineJ Ch<»t .<i national to-iit. t-urr i«'ulu3» tn.iy st.iiJ • 
nt'fdrd ahift ti>w.tr.J pt *>ror»& skjlls. Today's studi'nt:. m»iMJ f. 
driRunsC TAt <^ thp* ^i>ility ti> think < x it xt'.ti ly , tiOtvi* proM*«fr... 
AppJy knowledqe. utiii^** ti'chnoUj^y , undcint.^nrt data and w? it' 
ami coniRwnicato u f ! «'Ct i vr 1 y . Ko rteed to he tea^-himi ,j M v! 
tht-Ke thioqs. A nat lonai t«-Kt on basics '.'Ply nt.^y lirit 
teavhinq and Irarnimi in the protests skill atea:i. 

Any fffott t->warJ a n.itional ast5eyr>nont muat cons^iiiT thi 
t'xteht to whirh statOfi axe now aQnonninq. are eonvern"* 

at»t7ut th<? hufjt'' tlfir.jnd on \n&trut*t Jonal tijae that the n.'j7>, 
layers, of tests; and tho f rfquoncy oT v-urrent asist't-sat'nt T)v - 
lequirc In Kichiqan. we hAVc yi'arly atatewnli* t vf^i y-y up i i 
testing itj N**adin<5. Math anii 5'.v-ienL€ at s**veraJ 
intf»rvAli.. Many di&tru'ts f»n»ploy coropetency rest in*^ fot 
araduat jor. . teai'her-d*>voIf>p**d tOfit« Art* tmed routin#:ly. .i; 

afr the various national test*; currently in pMt-c {NAM. 
ACT, '••AT? . 

Cofopar 1 Jionti oi te^t data betw»H»n fichot^I districtr.. strtt*»K. .iiw 
even other countri^.-r. .hh often unproduct i v** t>*><*.4ijs*' thi»v J 
fMit take rt-qipnal ^nd othfr J i { 1 t-t 4'n»*f»u »nt::» account. J.r 
t»xjifnple. url-an {economical Jy and educat i ^ nai i y d » s^idv.int .»ij<' J 
ethnic differrnres. fcw^-r educ»tionai r*"iourc*>f>t and salju»i.i», 
(mirfdlo cl.iu!^. proftvtiiona] . ^v^*^\t»r educational ri-:,».u».i 
r.chooi d J At rift compai iji<?n>i df-aifmi.t rat e the titvd tn i1t»v*-? .j 
qii i(l«> i 1 nps for i nt erprt*t inq and ti^^inti teKt r»^*suirfi ii#»yon.i r 
ot Aqv^rt'^at*? Er^i^re t or>paf i tions "how weli" a »J i -it f ji ^ ; 
Joini^. We suhst r lt^c• to the f»hifo-.nphy th,it aii r.t iiH< r^t s r 

J munt iea rn e;js«'nt si< j ' 1 s . cmn*. «*pt t* ^nd pr<>i"esf.es . i« 
r"!..uit« f^notJid be ttj ex.tfaine the otcnt t« wr.i.:. 

<'iiuc.»T 1 on.t ) i'quiTy ,init tpi.ilrty of I<*,irninq har. t»reti ^n^..^T■v * i. 
ov€*r r > 2 1 h in a p.Trt n-u I »r d » st r i rt f*r lit , ni-t in 
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TP»© Honorable Dale Hxltleo 

cwmparidon to others who My hAve 6tax:tfi4 «s ditterenl 
point on the ucaU> du« to certain rt<*vant«<jo« or Jis^dv^jnt.^QOK 
they tvavQ. 

• t>cvelopiaent ot a not ional curriculua, or a set ol nat ional 
qoalii and object ivew for <»dacatlon, muKt involv© collaboration 
fcilth not only tho leading subject ar«a spc^cial iats and 9<»ner<sJ 
curriculum dovelopmont experts, but aXso with tha local c&vhool 
district teachers and adtplnititrators who are most closely 
associated with tho success of our ^ucatlonal system. 

We appreciate this opportunity to conviaonicate our viewB on thp 
national tf»stin9 delL>at«». Pleas«» do not hosjtatc to call on Widdl<? 
C »t ICS for lnfori»«t ion and as&ist^nco as your Coiw5itte«» d*'bj»tes 
th){. important issu^. 



I 

Mtcha&l A. tkiulus 



Vioqislat ;v«T I nf orwat ion l)i rev t ^ r 
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Houghton Mifflin Compan^ 



O'H? Bedcof* SJft^'! ^\)i>:o'^ Masj».K r^.-M'^'-^ OJ' 




STATE MENT OF MICHAEL MELCPY 
ON BEHALF OF H OUGHTON MIFFl fN'COMPANY 
StiBM ITTEp_ FOR_INCLUSION ^ THC RECORD 
HEARING 6_N_^ _N"AT ZONAL TESTING _?ROS' AND ^ CONS 
BEFORE THE SUBCOMMITTEE" ON_"eCEMENIARY. ' S£CONr>ARy 
ANT) VO CATIO NAL EDUCATION ^ " 
COMMITTEE ON EDUCATION ANlf LABOR 
^ ' M ARCH T4 . " 1991"^ 



Mr Chairrtan and jr.err&ers o£ the Comrnittee* name is 
Michael E Melody, I am Senior Vice President for College and 
Test Publishing for Houghton Mifflin Company. Hcughcon Mifflin 
Company is a publicly held* Massachusetts corporation which has 
been located in Boston for over 150 years. The company is a 
major publisher of textbooks and other educational materials* 
including assessment instruments, for schools and colleges/ 
general interest and reference books for adults and young 
readers, and computer software for educational and business 
applications, for both domestic and international markets 
I welcome this opportunity to contribute to your important wolk 

We employ over X*400 people in Massachusetts and about that 
many more throughout the rest of the country. Houghton s 
annual Massachusetts payroll is more than $45 million, and 
spend over $125 million locally with freelance writers, artists 
and photographers, printers, paper merchants, trucking 
companies, and other suppliers of goods and services each year 

Among our employees located elsewhere in the U.S. are nearly 
400 people at two locations in Illinois — our midwestern 
regional office in Geneva and The Riverside Publishing Compary 
in Chicago. 

The Riverside P'jbiishing Company » a wholly owned subs;i:ary 
of Houghton Mifflin, publishes standardized tests in a variery 
of formats. Perhaps our best known program is the Iowa Test :t 
Basic Skills which was first developed in 1935 under the 
direction of the staff of the college of Education at the 
University of lows in Iowa City. 

Houghtcn Mifflin first published the Iowa tests in 1940. 
thereby making them available to schools nationwide. Our 
relationship with the University of Iowa, now through our 
subsidiary The Riverside Publishing Company, is one which r.as 
flourished for over fifty years. Millions of dollars and 
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volu^es.cf research data have been .nve^^ 

fhirtl^^tlnf pJo^rarfn^S^/ef ;rreep%. up .o^date in order to 
meet market requi^e^^^^s . 

in addition. Riverside publishes 

attention- 

assessment program. ^''?!H m„??!Dle- choice tests- This is a 
deficiencies of such tests. 

JIIIoScI assessments and portfolio programs. 

higher ocdet "'ii^.'SSo!?; ^SS ^hl luSl an' 
fotosts. such ''."""'P?"-;"!";- , -l«;f<,tm.iice- focmst, 

The oroblems of education m America today can not be 
attcM liSustically to Jff^^«^^^3^"i^^fafdized tests 
information derived ?""f",J^|tJn """ow 5ln a student or 

are "o^the only source of made that claim. To do 

a school system is f ^^^est and undervalue the role 

T.fieZlirill tSl Kll°syltir?o^provide relevant and 
accurate information- 
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b^liIvfrHfr^Kf "^^n used for many purposes. We 

c^r»nX^he ^'^^ important use is in diagnosing Individual 

strengths and weaknesses to iniprove the instruction of that 
^!^«ni;«c r*"^^ 5*" "^^^ " determine how an individual 

?n2?.t r "Students nationwide as well as whether that 
oS?I^tfv^^ °' national instructional 

cr^fS .^"'^'''"^"^^ ^"'^^ "S^'i by the local and 

state education authorities for their own accountability 
purposes, to evaluate the results of the effectiveness of 
instruction, and to evaluate and improve curriculum, 

„ kJ^'k*^^'"'"® ^^^'^ ^^^'^^ ^"^^ effectively anj appropriately, 

publishers provide specific guidelines and suggestions for the 
interpretation of test scores and how to presint the scores^n 
ways that both pupils and parents wni find helpful? hoS to 
diagnose specific strengths and weaknesses; and how to use -est 
results to improve classroom instruction. >V pru^ide in-service 
support ro teachers and test adniinistrators . These effortflre 
in accordance with the Standards for Educational and 
Psychological Testing and the Code of Faif Testing Practices in 
Education. (Copy attached) ' tr:._..i»j j-ractices in 

m-.J^^"?"'^! school systems, parents, and the public of the 

ls^:?5„'^^'*^^^.^^'"^ publishers also document the 

reliability and validity of their tests, and support their tests 
through extensive research and development efforts Riverside s 
test authors are eminent educators and psychometricians with 
expertise not only in test construction but also in curr culun 
fmnT^frT;^^; instructional application, and the psycholog ca 
fTFti^I ? testing in school and university settings, o-^er 
JlSrJ^h M '^t^^'""^ University of Iowa and over 300 

research studies have produced information about test 

S?ver^^rfl'r/^f the foundation of excellence on which 
Riverside s tests are built. 

These same standards that have been applied to 

^ci^«^^;^^°'''?.^?^*^'' "® appropriate for performance 

assessments. It is important to stress that these formats a-e 
complementary. Performance assessments should not be cons-'^ecC 
a panacea, nor as a mutually-exclusive alternative to 

M^Jtl'E ^^""^ serves special purposes and >'.as 

unique advantages and disadvantages. f f ^ auu ..ds 

Performance assessnier.ts are not new Teachers have 
:l^^i^t°''t evaluated writing skills by grading papers tha- 
students have written. Riverside s Iowa Tests of Basic SkV-'s 
and Tests of Achievement and Proficiency include Writing 
supplements for grades 3-12 that have standardized this prcc--,- 
by giving students a standard set of topics and tandard ' 
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guidelines for completing the essays. The essays are then 
scored using a standard procedure. Since all students follow 
similar directions and are graded in a consistent manner, the 
score for each writer s essay can be compared to national norms. 

As mentioned earlier. Riverside also is developing a 
statewide performance assessment for Arizona. It will 
complement the Iowa Tests of Basic Skills, enabling assessment 
of progress over the school year from both a national and local 
perspective. The new test will be administered to grades three, 
eight, and twelve in the spring s^nester. It requires written 
responses to questions, including short answer and essay 
responses in the reading and writing sections, and graph and 
chart constructions in the math section. Arizona teachers will 
be trained to score the new tests, to assure that the results 
will be fair and comparable. Dr. Monty Neill of FairTest spoite 
positively of this project in his comments. 

While it is true that performance assessments are not new, 
it is also important to point out that the use of this format 
for a wide range of subject areas in a high stakes context, 
where the results are to be used for comparisons of individual 
students and for system accountability, is breaking new ground 
Accordingly, these expanded uses for performance assessments 
shoul'* be approached very cautiously to assure that they do meet 
the essential requirements of fairness, validity, and 
reliability. 

Which format is used, of course, begs the question of what 
is being assessed. It is unfair to criticize multiple choice 
tests for not providing information on how a pupil v performing 
on national standards when those standards do not yh^ 'Xist. In 
the absence of a national consensus on a standard for what 
should be taught and tested, standardised tests that compare a 
pupil to students nationwide have been the only objective and 
accurate appraisal of achievement beyond the local curriculum. 
We are now in a transition periods during which a consensus is 
growing that minimum national educational standards are 
appropriate and necessary. But that consensus is not yet 
attained nor has it been tested. 

Is there a national curriculum now because publishers sell 
their texts and tests nationwide? We believe not. There still 
remains considerable diversity around the country on curriculum 
and the scope and sequence of instruction. Recent advances m 
electronic publishing also make it far easier and more 
cost-effective for us to accommodate the demands of our 
customers to respond to their needs. It also makes it easier 
for us to respond to rapidly changing national educational 
movements and the resultant market demands. 
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To provide one example, in 1988* the Bradley Commission on 
History in Schools released its report, "Building a History 
Curriculum; Guidelines for Teaching History in Schools. ' 
recommendations intended to improve the teaching of history as 
the core of the social studies. Houghton Mifflin Company 
responded to the Bradley Commission's guidelines with Houghton 
MiJfUn^Spcial Studies 1991. a kindergarten through 
eighth-grade texthook series . Through this program, Houghton 
Mifflin met the Bradley Commission s guidelines and put its 
theories into practice. The books have already been adopted by 
California, Arkansas. Oregon, Indiana, and West Virginia, but i 
would be premature to say that it has nationwide acceptance. I 
there were to be a national assessment on social studies, would 
it reflect the Bradley Commission s guidelines or traditional 
social studies instruction*' Would two assessn^ents be needed, s 
that systems that have adopted the guidelines will not be 
disadvantaged' Would assessment be delayed until there was a 
national consensus on the Bradley guidelines? Shoujd rhere be 
national consensus on the adoption of the Bradley guidelines"' 
These are very practical considerations that must be part r^ 
dec is ion -making process toward a national examination system. 

As we know, similar situations exist in mathematics and 
science as the result of national standards being developed by 
the National Council Teachers of Mathematics, the Mathematics 
Science Education Board, and the American Association for the 
Advancement of Science, as pointed out by Mr. Shanker . 

Our authors and editors do go through a consensus 
process in developing our tests We consider: 

* current emphases in instructional materials, such 
as textbooks, 

* recommendations from national curriculum 
committees and teacher training specialists, 

critical evaluations and suggestions of classroom 
teachers and school administrators who use the tests « 

* social utility studies in relevant curricuiar 

areas . 

* comprehensive item rryouts and research studies * 
determine frequency of error, particularly in language and 
mathematics < 

* independent reviews by professionals from a 
variety of cultural groups to assess the fairness and 
appropriateness of items relative to demographics, race, ^nd 
sex; and 



- 5 - 



360 



* topic Importance, biased on author itative 
judgwenCi instructional trends, and public opinion. 

Other test publishers go through comparable processes. The 
diversity and dynamic process of publishing instructional 
materials also mitigate against homogeneity and minimiEe the 
chances of a close alignment of texts and nationally-available 
test. Factors against such alignment include that fact that 
there are many more textbook publishers than there are test 
pxiblishers, texts and tests are being constantly revised in 
differing cycles, and tests cover multiple subject areas for 
which there are likely going to be multiple texts from different 
publishers- Tests, however, may be selected to more closely 
correlate to tne local curricul'im, which may facilitate a school 
system s use of the test to evaluate pr:3gre3s on local as veil 
as national objectives. 

finally, let us recognize that over tne past r-^enty years 
there have been real gains in basic reading and numeracy skills, 
with the greatest gains being realised by minority students. 
Minimum competency is now nearly universal among high school 
graduates. That does not mean complacency about higher order 
skills or the condition of drop-outs, but there is caose to 
accept that there have been successes. These gams have been 
recorded on numerous tests, not just on standardized, 
multiple-choice tests. If minimum national standards are 
establishtsd, and if those Stan-' 'ds position our students to 
meet the challenges of an international economy, perhaps in a 
few years our assessments will show comparable gams to those 
shown on basic skills. 

To assist the Committee s deliberations, 1 am sutoitting for 
inclusion with my statement materials which The Riverside 
Publishing Company has developed regarding performance and 
multiple-choice testing formats. Although I can speak only for 
Houqhton Mifflin Company and The Riverside Publishing Company. I 
am confident that I speak for the commercial textbook and test 
publishers in general when I say that the industry stands ready 
to share our publishing expertise with the committee as you 
strive for a solution to the critical issues we are ail 
addressing. 



i909h 



6 - 



er|c ^'''» 



I 



361 



c 


0 


D 


E 


■ 


0 


F 


■ 


■ 


■ 


\ 

\ 
\ 


■ 


F 


A 


1 


R 


■ 


T 


E 


S 


T 




N 


G 


D 


R 


A 


C 


S9I 


1 


C 


E 


S 


■ 


1 


N 


E 


D 


U 


C 


A 


T 


1 


0 


N 


■ 


■ 


■ 



Prepared by the Joint Committee on Testing Practices 



Thf C\»dtf ot f ji»r Te^dniJ f*rd4tKf5 »n Education >tdies 
!tw ma^of oHiiUlii>f>i U> test tdfcers of profeawtmafs who 
iSe^eUtp Of uie «luu*{}pnjf tests Th^ Cti<Je ts meant fo 
jpply br«'>5d!y to thr liir ttl !«ts m education t«dm««w>jb. 
<n2uLatM>n«l a^ssmcnt. nlucatKmai and Mu' 

dent pf^Tem^M* The C<Aie »s rv)t <tr4>gned to cover 
tmrluvmem icftinj^ iutnsure ot ctfftificatton Irtttng, or 
other trpes of !e5t»nil. Ailhou|jh the CtxJe has reievantr 
To manv tvpe?* of fJuc^ttonil tests » direirtrd ptnirupilv 
3t pF^tteisaunAlt^ 4<\ifU,p^ \ests M>t.h those Mtld hy 
i«mm«rftwt t fVihh^hers used m toru^allv **Jmm(i- 
lere4 ttrsfrnt ,tr<j|{r3fn* Thf i'lxle r* mil rnr<'ndexl lo 



.tiver tt*«t5 miti^ by indfvfdKii tfaihtrs U\r us*- m !iH *f 

The CjhJc address» the roirs trf te^t »JfM'I ipc r*. jmJ 
le^t us*^* separately Test users 3re re«»r'«* *h<* *^hx^ 
trstSi commiss**.>n te*t J^-vfopment wr^Kti v»r ttuK« 
Ueuvtfm <»n the ha*i» ^rf {e**» stores TeM jjfwrl<>rrrv jr*- 
pT' fr*<f wHii actualiv ci>fv&ln/4l test* « »»eH is th»vs«- 

! f>*jfu"»e» Uir (WtH'uiaf testing jwtigram* Thf 
-njv ♦itifuirv overlap a* when ^ «3te etJuv,»tnm n» \ 
*'.mm»(iwtfn\ fesi 4e«eWpm«:nt iertiie*. x-li f«»lM«*% -^ ii 
. •'^ff^^^ the teM ievelopment prttieiw anvf Tijf^ \ Ji 



wd^HJO »n i««iftj#imu Ami J vvrU'Twnt AfU ihf ^ibmT>i^ >neeih 



t'^.u\jtigr^ ywjst Kite ih« Aj^Mmem a5 r"i*i«»* 
-tM Utmnvfire --^ feiitntt t^tt^e* .Mititn^ VJrr\* 



Bill 
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Code ol TcftuHl PnctKr* m £4uation 

The Cod? pmenu lundards k>r educauonat test deve t- 
open &nd usm m i&ur mss 

B. JntrfpT*fing Scorn 
C Strvtng for fsirnrM 
0, !nh>rm>nfi Tr$t Taker* 

who f ndofv the Code it^mmit ihenaeK'e^ to u^uard^ 
mg the niJhts ot test l^lwf* *»* '^»kiwmg the pnnapJe* 
tisteO The Cvfiv A mtendeil to tic cofuatent with the 
ftlev^t p*n& of th« 6'fiEm{^«ifvff for E^hcationai and Psy- 



the Coiie differ) from the StAHd&r^ m txHh dudienir 
and purpoie The Code u m««fit to be under&tood hy the 
general public it i5 tiitiited to edvcatK>n«l te«U and thr 
pnnury focus i» on those osue* that affect the proper 
as« of tt*ii> The Code is not meant to add new prmuplev 
o^w^idabr* : •.^^?e m the Standards or tocKaARe tht 
meaning of th? Standards- The goal i& rather to rcprrvnt 
the i|Hnt of a selected portion of the Standards m d wjv 
that IS meanLRgful to teM taken and or thvtr parents it 
guar^sans li a the hope of the joint Committee fhst ihe 
Code tvtll aiMi be fudged to be coft&»tent w«th cftMrn^t 
code« <^ conduct ai^ stAndarib of other p4xiic&»ioiut 
groups who use educattonai te^ 



I TlKtiiiljtfnhpiddpfW*^ftetafofqttttoPttei 



L 



K tJvftnv what fjih muMm tnd wh^t ih< te»t %fK>utd 
t)e used fut Descnhe thr popuUbtmi^t {(tf itrhwh the 
ir9t «t tp;m>pnat« 



2. A44-unteiv rxprnenl fhv sha}«rttntite& tisefuincu^ AnU 
hmttaliooi ot inu for their mteroled pvtposn 

X Enptain rrlrwit mta>ur«mrnt corK^i» n«or$ur> f<n 
ilar^tv ai tt» Ir^rl t>{ 4rt«(l th«t t> ap^upnate to* itis 
•nftnded »udi?nctt4> 

4. Oncnbv the pnxm !v»t diwrtofment f ipto ho«* 
ttw cimtent and iki\i\ <o br tened wm letected 

$. Provide rv»dWK* ihAi ifit tfsl nwrta tts mtertded 
purpowti 



^ Kuvidt ettNr » jpi t9 int at?vf uniptes or compktr 4i^p«« 
of te«t <}u«itmm tfirrrtKms tnsMvr »h«rti mmuh. iti^ 
vort rtpom u> auatificd osen 

7. fndtcatf the rutiire of the cMdencr oHaincd coftcemmg 
ihe appfopnatencsa of eadt tcsi ior groups o< differ^t 
imial cthrkMU (ti ungutMic backgrounds who ai* Uw^^ 
brtrsted 

% Identihr and pubtt^h any qwaalued Uulb n<«dcd <o 
admmum each test and to tAKfprrt tcurci cofrtctly 



Ite t0O9 should KS«ct tola that aeet the purpose 
foewh^ they ■« to bi uM» ^nd Ihitf «c apprQ^rrutt 
loe tite toifiidad trtf^&limg p(^i«U0ct&& 



Test Dam ShosSd} 

1. F(nt deftrte ihc pitfpmr fur testing and the p«f^lair*,n 
tt» he tested Then, »eJeii a te>t *o» that purpine an^J ♦h^t 
poputMmn bued i>n a tht»rough review <>l l^r j^attjhir 
>n(onnatM>n 

2. Investigate pntentialiy usefut tourers of inKtrm^ti"" 
addtttuo t<» test stotes. tu ^(.irrutK^e the mfumMt'w" 
provided bv lest^ 

3. tte«) tfie malenab pmvt^ed h^r te^t developed Artii 4«»t.j 
usmg tests tntr Mrtmh umtear iir moumpfete tnN>rm<»t»'n 
iiprovHjed 

4. Be^umr famtiiAf wtth huw i^tven the teM ^r^vt 
uped and med tmt 

5. Read ir^ependent r>«» ^rtaa * t&i jjul <*< :^oVM*«t7 
ittrmstMe measures txiofc for evtdente ?e^v<rr J ti. \ur 
piin the cUtnts oi tnX dewtopers. 

d. EiLUPtne tpecrtnen letfc dfS»iu«ed te*ts ««r • 
U^Aestttm& dtrectKMHk annrer thertK manuitv and v"'«- 
teporti tsefore selecting a tcu 

T. Ascertain whrthev the tc^ ccAtent and f>om>s gr'Klp'^• 
or tompanson grotipf i> tn approprute for the tnt<rti)rti 
test takers. 



Secret and use ooty those tc^ for whKh the <a»U\ 
needed to administer the tc$s snd mterv^et s.f^tr\ . . 
recth are a«ttdablc 



*H*nv tlw fUUtnent* rhc Code refer to the trlerfioe erf not 
tn( te»ts Htiwevff in cusiasused te»t^ pra0*tts test dewkifv 
(ft iiv mgificd In consmKt r»w tata In t^M•e utvetukk ttte 



trtt deveio^cneni ptocvM thouid bt tles^netf to Fwtp rn!K«'r ■ 
ilu (umplned tots miU be m unnpik^K* with the K «vKc 
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Test i}cv«top«n Should: 

•S^vnhr test per^i>rm4n*r ^frarU jrrd Av\ura(f U 4lv> 

.»f .«Mt^par»t.«r> ifft'up^^t iht djtvs fhr JjU ^tm* iJ^tn 



tj. Ham uten {(> d^fiiJ v{Vtth. 



iM^r^opruif iM une M^h vt^nr* with the tfst 



i^fituptii and tft* lifTuutHms ot thf wuff* 

to, iMTffjtfrl Hf>rts Uking into j4.»,'nini anv m*t»»T J-Mtf 
iHvt * tJrtwrrn ifv norm* *w »,rfnjMT-v>ri jItuup> jnj 
AtudJ test ukets. Also t#fct mtu 4m'«nt jn\ J "f t 

mended ^v tfvc t^i d«vrk>prf unlr^ f'^tJcnv^* 
• Htff^ t4j »uj»pen thf mfenitrd i/» 



I3l < >^tam fw4fn\¥ (o tictp ih». 



Slnvinj^" for fnirnoKS 



Test dnvlopm should rtm« to mftkt tc^ thtt «v tt 

fvr a pos$»bkt for tz$t tifcfn d( drffitrem nres 3?iv 



Test utei nlMuid cckct ttftB thtt been devrf 
oped m wfn £htt Attetnpt to nakf th«m fair u 
rosuMr GTK tiksfi of <lif{«reftt turx fth 
fMf fa itkBiMtihi ^ Of t i t nd iappatg coodrtioTtt. 



14. RrviTw An4 fr^ite l€»t kju^strmu And rf iiled matf ruU 

ti. JdvcsttitAtc the perfonuRce af tesi i^tn *4 d^ffff^rx 

wtficitnt MIX irr jv«it»b{« 2sm^ fuvie^um itut ht\p 
to mtuiT !h*t 4vffe»TiKri m ppHorrodriif »re ff^Med 
prmAnf> to (hf vkttU iifHiff AMCumeftt r^thff th«n tu 

Itn With lui>d»i^ai>p.T))fi ionJitiuru. W«m usen t*f 
potfntjaJ ^r>^trml ift u»in^ itandard normn *tth 
i ftfd tests <if »d«>m jrt fa! i<»R r'oirdufr* th^t wutt m 



Test l'a«r» Should: 

t4 Kv»Ju«*e thf pTfKpOutn u*eu M tf»{ Jr*ri<'f>rr^ 

IS. Rfvtew thf prrfornvkn^e «f tett Ufcfnof tSttte^rtM t^K^y 

t tfnt stff 4rf A^attiHe £v;>liatf the e»tf nf u> M^hu n 
PfTh-jrnunkf difference* nuy h*vf heen K .r^p 

prttprute k hfir^'tfnsttct of (be le^ 



IS. When nei:c*s*f> *nd teawHe. u»e Apprtjfin^tri^ n»%; 
Hfd fr>rmi ^ tesu or admmtsinitftn pfv^f dufr> u,i 

ntTmi mth 4:Anr tn the light of the minSi(t\jU'n% that 

*f tf ffU*Jf 
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TcfS P mio ycf i or Tc« I'lm Sbooltfi 

17, yiihtn « t»»t « optfiiTul ptw^ trtl Ukcn of Itietr F^rtnlA guanJun* *tlh tnfortiuUoo help <h*m JoOac •ftv^htr ^ 

f>ov*4f tMi «*kn» tht mfofwalioti thn ne«J lo be wnilw ihr io^mgc ol the jeM. t^« tvprs o( ^ur&liun j 



Vfitkr Hi9t c^oafUnc*! ten dMkpm biMt dtract ccniro) of t«st» knS tm moftM, iMtr oltMt csicuotstAjicM 
tmn hM Md) ccctroi Wbidmr r«ip Iw control of tntft va ttft mm ibottU Uk» tht dncn t>cU 
bcio«L 



Test or Tcft l^tm SbooWt 

19. Ptrn^ !r«t ulcm of their pOTno- ffuitntuMtt with tnfonnMion About n|thl» tot uktn m^v h«w le obtAtn ^ «>p»n • .i 
tc<t» and compirttd tnsMr thecti^ nruke WX^ tme test* fescorrd or cftncct Kom 

90. Tri) Ust tkkert or ihev puvntft> ffuartfiam Now Hong iexwp« v^U h* kept on file ind indicate ti> wham Ai>a arnkr wh<7i 
cntuiBstviccs left tasn% mU or «U not bt f^Ciit4 

21. Descnbr (be prof»durr« that left taker) of then parents gtjAfiJcans may use to iT0atet compUtntsi and ha>v rr«»h;^m^ 



Note Th6 ixwinbcnhip of the Wortemtf Croup thtt dcMttoped the Code of Testtng Prsrticies in BducatKin ^^li or the 
/o»m Commmw <m Tesbitf Praciire* that ^utdcd the WoHung Ooup w«s as ioHoM. 



Thcodp«P Bartetl 
JohnR Bcfgaft 
Esther & Dikawnd 
Rich*n5P Dunn 
lorrvneD Cyde 
teyioood D fmeJcr 

iCttchiff. KTTP wd Ch*H 
Code Wotiantf Croup! 



Edmund W Cordtm 
lo4fUC Hanseti 

C«oc^F M«dius 
lCo-ch*JT, ICTPi 
KcvtnU MeerUod 
>o-£1Wn V Ptrtt 

i Sofomoo 
l<^T Stcwatft 



Ctfot Kehr Tittfe 
(CoNchwr. iCTPi 

Nichftel ) Tieky 

EWbn BoJta* and Wayne 
C^mtfk oi the American 
PiV€hok0c»l Asioiatutn 
teivtd as »UH ta»c^5 



MdiUonft) co<M« of the Code may t« ot««sM4 fnim the NHwaaf 
Couivtt Oft PlkaauiemtM m Edifcarav. tHU Snefttsnith Scrm 
vw Mra»Kiflg»A i) C 7S03it tsmm a» («m 
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